Editor’s note: This article is part of Just Security’s ongoing coverage of the Taliban takeover in Afghanistan.
In the wake of Kabul’s collapse, experts and government officials need to answer a pivotal question: why did the United States fail to build a more effective Afghan military? In its misguided focus on training and equipping, the United States failed to prioritize a results-based approach. To avoid a similar fate with other counterterrorism training missions, the United States needs to invest in and empower assessment, monitoring, and evaluation.
Since 2001, as part of U.S. counterterrorism planning in Afghanistan, successive American administrations have prioritized building the Afghan National Security Forces (ANSF) to take over the counterterrorism mission and facilitate a measured withdrawal of American forces. But the U.S. military’s approach was flawed from the start. The United States never conducted an assessment to determine the type of military Afghanistan needed given its human capital, geography, and adversary. Its approach failed to identify clear metrics for building such a force, establish a systematic monitoring and evaluation mechanism, and adequately analyze how Afghan security forces would hold up in direct action against Taliban fighters. These gaps in analysis were laid bare when Afghan security forces completely fell apart in the face of Taliban fighters, only four months after President Joe Biden announced the U.S. withdrawal.
Twenty years later, despite the United States’ $83 billion investment in the ANSF, security in Afghanistan has progressively worsened: the Taliban controls more territory than at any point in the war, the number of effective enemy-initiated attacks has steadily increased, and fear for personal safety among Afghans has never been higher.
The United States dedicated a significant amount of money, time, and attention to implementing training and equipping units, while less than one percent went to collecting the most valuable information – how the effectiveness of soldier and unit performance changed as a result of the training.
Mission Failure: The Reconstitution of the ANSF
Capacity building is extremely challenging in conflict zones where staff turn over rapidly, no single U.S. agency is in the lead, and chronic insecurity makes measuring success difficult. Afghanistan presented the added difficulty of conducting capacity building during active combat while dealing with the challenges of coordinating the efforts of an international coalition. Persistent insecurity severely undermined reconstruction efforts, forcing the United States and NATO to continuously substitute for the capacity and capability of the ANSF, which often detracted from the capacity building mission as well as the accompanying efforts to monitor and evaluate it.
There were many flaws in the way the United States measured the effectiveness of its efforts. While the U.S. Defense Department (DOD) is required by law to conduct assessment, monitoring, and evaluation of its capacity building programs, whether this is carried out and the quality of those efforts is not always guaranteed. Standardized methods to measure results did not exist in Afghanistan. During the 20-year effort, operational assessments from top U.S. military leaders exhibited entrenched optimism, which “bore no resemblance” to conditions on the ground.
In seeking to evaluate the ANSF, DOD changed assessment methodologies at least four times, resulting in varied, often contradictory conclusions about the quality and readiness of the forces. As a result, reporting tended to elevate good news and success stories over data suggesting a lack of progress. Criticism of the various assessment mechanisms included: a paucity of personnel able to execute the assessments and verify reported data, poor quality of the data collected, and improper metrics that assessed program progress (dollars spent, equipment provided, soldiers trained) rather than longer-term effects (battlefield performance, leadership, corruption).
The Writing on the Wall
In 2008, Congress created the Special Inspector General for Afghanistan Reconstruction (SIGAR) to provide quarterly reports to Congress and ensure independent oversight over the entirety of U.S. reconstruction efforts. In late 2014, at the urging of former military and civilian leadership serving in Afghanistan, SIGAR began publishing lessons learned reports to capture the progress and challenges.
SIGAR reports repeatedly expressed skepticism of the ANSF’s capacity to provide its own security and maintain facilities and equipment provided by DOD. In March 2014, General Joseph Dunford presciently warned the Senate Armed Services Committee that if the coalition troop withdrew at the end of the year, the “[Afghan security forces] will begin to deteriorate. The security environment will begin to deteriorate, and I think the only debate is the pace of that deterioration.” In 2018, then-Afghan President Ashraf Ghani estimated his army would not last six months without American “support” and “capabilities.” In its most recent assessment in March 2021, SIGAR concluded that the ANSF could neither protect the population from insurgents in large parts of the country, nor could it maintain equipment, manage supply chains, or train new soldiers, pilots, and policemen; the ANSF was “nowhere near” achieving self-sufficiency.
Beyond Afghanistan
This broken process didn’t manifest just in Afghanistan – it persists across U.S. counterterrorism programs globally. In Iraq, fewer than 1,000 fighters of the Islamic State of Iraq and Syria (ISIS) launched a surprise attack on Mosul in 2014. In response, four of 14 Iraqi army divisions – military forces that the United States spent years training, arming, and equipping – rapidly crumbled. Approximately 30,000 soldiers dropped their weapons, shed their uniforms for civilian clothes, and blended in with the fleeing masses. No one predicted that Iraqi security forces would disintegrate when facing a few thousand militants.
In addition to major counterterrorism capacity building efforts in Afghanistan and Iraq, the United States is involved in training and equipping programs in approximately 80 countries throughout Africa, Asia, and the Middle East. Based on my interviews and forthcoming research experience, in each of these cases, the U.S. government is not making results-based decisions on training mission effectiveness.
There is also a growing body of academic research that links military assistance to increased terrorist activity. The provision of military assistance has been associated in some cases with increases in terrorist attacks, fatalities and casualties, and support for terrorist groups, as well as longer terrorist campaigns and disincentivizing partner countries from disarming terrorist groups.
Mali, for example, has received significant U.S. military counterterrorism assistance since 9/11, yet Malian security forces collapsed when faced with a terrorist and separatist onslaught in 2012. Shortly thereafter, the military launched a coup that displaced the democratically elected government. Since then, two more coups have occurred, each led by the former U.S.-trained commander of Mali’s counterterrorism unit, Assimi Goita, who has since become the nation’s president.
Adaptation Drives Effectiveness
In each of these cases, the United States has failed to leverage existing assessment, monitoring, and evaluation (AM&E) practices to better understand both intended and unintended consequences. The U.S. military lacks process and prioritization for measuring capacity building effectiveness. Using the military decision-making process to plan and execute, the DOD tends to view its mission as linear and finite. By contrast, the U.S. Agency for International Development (USAID) manages programs that are cyclical and ongoing. To ensure efficacy, USAID’s program cycle seeks to plan, deliver, assess, and adapt its assistance. The critical difference is in the final step: DOD focuses on executing the order, while USAID incorporates the results of its AM&E to improve program efficacy.
Congress took a monumental step forward in reforming how DOD conducts and manages its security cooperation efforts in the enacted FY2017 National Defense and Authorization Act (NDAA). The NDAA dramatically increased congressional oversight by mandating the assessment, monitoring and evaluation of security capacity building programs. Because of DOD’s lengthy planning process, however, tangible changes will not be realized for several years.
The FY2017 NDAA also included a crucial development: the creation of an in-house AM&E workforce at DOD. The Defense Security Cooperation University was founded in September 2019, to uphold a common knowledge base across the workforce, mandate common professional standards through a required multi-tier certificate program, and prioritize continued intellectual development. To further bolster this in-house AM&E workforce and minimize dependence on third-party contractors, the NDAA mandated the establishment of teams dedicated to AM&E within the Office of the Deputy Assistant Secretary of Defense for Security Cooperation, the Defense Security Cooperation Agency, and within each Geographic Combatant Command.
Investing in Results
To build another country’s capacity to counter terrorism, the United States must improve its own competence to assess, monitor, and evaluate training programs. DOD should further invest in three crucial areas.
First, DOD needs a culture that empowers its workforce to go beyond reporting. Promotions should be contingent upon leveraging AM&E to improve program performance, rather than upon who manages the most expensive programs. The change toward results-based learning needs to cascade from the top down – specifically, the Deputy Assistant Secretary of Defense for Security Cooperation should be appropriately staffed and resourced to ensure programs institute and empower DOD’s AM&E requirements. The agency-wide perception that program evaluation is used as a punitive tool leading to budget cuts or other adverse repercussions needs to shift – no evaluation finding should be viewed as negative, but rather should be viewed as guidance for course correction.
Second, DOD needs to prioritize rigorous program design. DOD should place emphasis on a thorough upfront assessment, during which program planners analyze the situation on the ground and assess a partner nation’s needs. If these components are properly captured, the program will identify more realistic goals and have a better, smarter design. In addition, DOD has faced criticism for not prioritizing impact evaluation. Measuring program outcomes is critical to identifying whether a given program has achieved its desired effects. By identifying successful programs, DOD can focus on adapting, replicating, and testing those programs elsewhere.
Third, DOD needs a robust yet flexible feedback loop for its programs. Leadership needs to underscore the importance of iterative learning; however, the current 5-year planning cycle and congressional notification system do not afford required flexibility. In late 2021, planning is in full swing for FY2023 programs. Once in motion, the system does not allow for change even though a lot can happen in three years that could warrant adjustment. Furthermore, while DOD should continue investing in independent research on how to improve assessment, monitoring, and evaluation of security cooperation and capacity building programs, it should also incorporate findings from the growing field of academic research in this area that already exists.
Without a fundamental shift towards accountability, other recipients of U.S. military assistance stand to suffer Afghanistan’s fate. We should always have an answer to the question: “What did we accomplish?”