Thirteen years after 9/11, the United States Congress appeared poised to begin the long overdue process of reining-in the intelligence establishment’s runaway surveillance practices. If interpreted faithfully (a major “if,” to be sure), the version of the USA Freedom Act introduced by Sen. Patrick Leahy (D-Vt.) in July would end the bulk collection of Americans’ phone records and nudge the administration toward greater transparency. Incredibly, it managed to win the support of both the Director of National Intelligence and the ACLU. A few key holdouts in the Senate intelligence committee remained, but the bill’s chances looked good.

Then came ISIS. Following the group’s capture of territory in Iraq, its beheading of two American journalists, and its calls for followers to launch attacks in the US, some American lawmakers claimed it would be irresponsible to ratchet back surveillance authorities in the face of a new terrorist threat. These lawmakers – including Sens. Saxby Chambliss (R-Ga.), Marco Rubio (R-Fla.), and Lindsey Graham (R-SC.) – were never supporters of USA Freedom, and it’s not clear that ISIS has actually changed anyone’s mind. But the issue sucked the oxygen out of the Capitol and consumed the miniscule amount of time the members left themselves to do business before the mid-term elections.

My purpose here is not to question whether ISIS poses an existential threat to the US – I’ll leave that debate to others. What I question is the assumption that such a threat should weigh against surveillance reform.

The US has the benefit of hindsight in assessing which post-9/11 counterterrorism policies went too far in curtailing civil liberties without commensurate security benefits. We shouldn’t cast these lessons aside because of a perceived new threat; we should learn from them. At least two of these lessons are reflected in USA Freedom.

Lesson 1: The executive branch has a tendency, when operating in crisis-driven, “chalk on the cleats” mode, to secretly push the boundaries of the law.

This happened when Bush administration lawyers decided that the president’s Article II powers overrode both the Foreign Intelligence Surveillance Act and the laws prohibiting torture. It happened when Obama administration lawyers used a creative definition of what constitutes an “imminent” threat to justify the CIA drone attack that killed Anwar al-Awlaki, a U.S. citizen. It happened when both administrations concluded, and the Foreign Intelligence Surveillance Court (FISC) agreed, that the phone and internet metadata of all Americans were “relevant” to an authorized international terrorism investigation. It ostensibly happened (although we know very little) when the Obama administration decided it needed no additional congressional authorization to conduct airstrikes against ISIS. All of these legal interpretations were issued secretly and remained secret even as the US government acted on them.

To keep the lawyers honest and the people informed, we need greater transparency about the legal interpretations on which the US government relies. The USA Freedom Act would require the executive branch to release redacted or summarized versions of significant FISC opinions, which reflect and generally ratify the executive branch’s legal theories. Of course, this measure would not address all manifestations of secret law, even in the realm of surveillance. Most foreign intelligence surveillance activity takes place under Executive Order 12333, which operates with no oversight by the FISC. But it would be a start.

The USA Freedom Act also would require more detailed statistical reporting by the government on the number of people affected by specific surveillance authorities –including, for most FISA programs, a separate tally of U.S. persons affected. These numbers give meaning to abstract legal interpretations. It’s clear that the FISC endorsed a broad interpretation of the term “relevance,” but only the numbers can tell us exactly how broad.

Another potential solution to overly aggressive legal interpretations is to give someone the job of pushing back against them. Currently, with rare exceptions, the FISC relies on legal arguments and factual submissions from only one party: the government. The USA Freedom Act would establish a panel of paid advocates who could serve as amici representing privacy interests in significant cases.

Lesson 2: Bulk collection doesn’t work.

There’s no question that using metadata to link a known or suspected terrorist to a broader network is an effective counterterrorism tool. But collecting everyone’s metadata does not, and did not, add value. This was the conclusion of the president’s own independent review group, which included a former acting head of the CIA and a national security advisor to President George W. Bush. The group found that the information derived from bulk collection “could readily have been obtained in a timely manner using conventional 215 orders” – i.e., through collection targeted at suspects.

The USA Freedom Act would end the bulk collection program. To collect phone records on a prospective basis, the government would have to identify a specific individual, account, or personal device, and persuade the FISC that it was associated with a foreign power (or agent thereof) engaged in international terrorism. The government could then collect the records of the approved target and anyone directly connected with the target. The bill also would prohibit bulk collection of other types of transactional data, although the wording of these bans is susceptible to distorted readings, as some have observed.

Defenders of the status quo say bulk collection is necessary to preserve the data in the event phone companies (the source of the data) decide to shorten their own retention times. But a wholesale intrusion on Americans’ privacy shouldn’t rest on a problem that may never materialize. If, at some point, companies signal an intent to change their practices, we can have a public debate over whether security benefits stemming from longer retention periods outweigh the risks of mass collection on law-abiding Americans.

If anything, the USA Freedom Act may not go far enough in applying the above lessons. Most of its major provisions have potential loopholes. The Director of National Intelligence determines which FISC opinions are “significant” and how much to redact. The requirement to disclose the number of US persons affected by surveillance programs effectively exempts section 702 of the FISA Amendments Act, a huge program that captures calls and e-mails between foreign targets and Americans. The FISC decides when, if ever, it wants to hear from special advocates. The bans on bulk collection are imprecise in their definitions, a practical necessity that nonetheless opens the door to mischief.

Moreover, while USA Freedom makes substantive changes to only one type of surveillance – the collection of information held by third parties – the lesson derived from bulk metadata collection likely can be extrapolated to all forms of mass, suspicion-less surveillance. Dragnets don’t work for a simple reason. Human beings can’t analyze all that data, and “pattern-based” data mining (using computer algorithms to tease out patterns that serve as predictors) has shown little promise in the counterterrorism context.

What can work in theory – and what has worked in the few examples of intelligence successes the government has chosen to disclose – is surveillance of people who are known or suspected, based on objective facts, to have links to terrorism. Such surveillance then leads to the discovery of previously unknown terrorist associates or plots. This suggests that reforms to section 702 and Executive Order 12333 – neither of which requires suspicion of wrongdoing or security concerns – are in order, as well.

The rise of ISIS is no occasion to jettison these lessons. There are many risks inherent in placing the country on a perpetual (or at least indefinite) war footing. The potential for a steady erosion of our civil liberties, resulting in a permanent recalibration of the balance between privacy rights and security claims, is one of them. It’s an existential threat of another kind, and we ignore it at our peril.