This morning, the New York Times ran a story about an internal CIA study commissioned in the last two years that found the agency’s historic efforts to arm rebels have had a minimal impact on the resolution of a conflict. Indeed, the CIA found that those efforts were even less effective without what the NYT describes as “direct” American support on the ground.

Given the CIA and Pentagon’s efforts to arm Syrian rebels, the agency’s report raises serious questions. If previous efforts have been largely ineffectual, why arm rebels in Syria? And if the U.S. government is committed to arming the rebels and it knows that a lack of American support on the ground means the efforts are more likely to fail, why not provide that extra support? What would “direct” support look like given the fact that President Obama has insisted he will not commit American ground troops to combat ISIL in Iraq and Syria?

In fact, in a January 2014 interview with the New Yorker, President Obama discussed the report and the difficulties in arming Syrian rebels to fight Assad, using it as a reason not to engage in more expanded efforts (emphasis added).

Very early in this process, I actually asked the C.I.A. to analyze examples of America financing and supplying arms to an insurgency in a country that actually worked out well. And they couldn’t come up with much. We have looked at this from every angle. And the truth is that the challenge there has been, and continues to be, that you have an authoritarian, brutal government who is willing to do anything to hang on to power, and you have an opposition that is disorganized, ill-equipped, ill-trained, and is self-divided. And, in that environment, our best chance of seeing a decent outcome at this point is to work the state actors who have invested so much in keeping Assad in power – mainly the Iranians and the Russians – as well as working with those who have been financing the opposition to make sure that they’re not creating the kind of extremist force that we saw emerge out of Afghanistan when we were financing the mujahideen.

Without access to the full report, it is hard to know how the CIA is defining the effectiveness of its past efforts and what that may mean for current or future programs. Hopefully, this story is the start of a broader conversation about arming rebels in Syria with seemingly little military or political oversight on the ground. (Speaking of oversight, see Sarah Margon’s recent piece in Just Security on the importance of ensuring such weapons stay out of the hands of rebels who might use them commit atrocities.) Whatever the public consensus winds up being, it is certainly a conversation worth having.