Skip to content Skip to footer

Creating the Intelligence Bureaucracy

FBI Director J. Edgar Hoover, 1924-1972. Hoover's list of 200,000 "radicals" foreshadowed today's watch lists. (Photo: FBI)

What a difference a century makes.

One hundred and three years ago, then-attorney general Charles Bonaparte enlisted some private detectives and members of the Treasury Department's Secret Service (set up in the aftermath of the Civil War to ferret out counterfeiting) as special agents in his newly created Bureau of Investigation.

At a time when Congress was staunchly against any federal power engaging in political surveillance, its role was initially limited to investigating interstate crime and crimes on federal property.

Those limitations did not last long. The seed had been planted of what would become a massive intelligence and homeland security bureaucracy involving 1,271 government organizations and 1,931 private companies, according to The Washington Post's “Top Secret America” series.

How did we get from there to here?

The journey crosses a landscape of manipulated fears, as the established order confronted the “radical” ideas brought by immigrants, a series of anarchist bombings (including one on Wall Street that killed 33 people), an upsurge in trade union organizing, agitation surrounding the United States' entry into World War I, and the epidemic of strikes and race riots that followed the end of the war. The strikes of 1919 involving millions of longshoremen, stockyard workers, shoe workers, subway workers, steel workers, coal miners and members of the Boston Police were depicted in The Wall Street Journal and other newspapers as “Bolshevik” or “Soviet-inspired,” or as a kind of “terror.” Anti-union fever was solidly bipartisan, with the political parties stridently denouncing labor organizing as an attack on America and its way of life, and Massachusetts Secretary of State Albert P. Langtry denouncing political radicals: “If I had my way, [I would] take them out in the yard every morning and shoot them, and the next day would have a trial to see whether they were guilty.”[1]

This first Red Scare opened the door to domestic surveillance by federal authorities. In 1919, Attorney General A. Mitchell Palmer set up the General Intelligence Division (known as the “Radical Division”) within the Bureau of Investigation and put the 24-year-old J. Edgar Hoover in charge. Hoover drew up a list of “radicals” and, before long, he had an index of 200,000 names.

The list was instrumental in the massive 1919-1920 roundup of an estimated 6,000 immigrants and citizens, known as the Palmer Raids. Foreshadowing the post 9/11 roundup of “persons of interest,” detainees were denied access to their families and to lawyers, and many were held for months, with no charges ever being filed against them. Hoover's list grew to 450,000 names in the aftermath of the raids, as the General Intelligence Division of the Bureau conducted wiretapping and break-ins. Red Squads formed by state and local police, along with the 250,000-strong volunteer force known as the American Protective League and its successor organizations, also engaged in the hunt for the “enemy within.” George W. Bush-era attorney general John Ashcroft would later draw upon a long vigilante tradition of enlisting citizen spies when he proposed Operation Terrorism Information and Prevention System (TIPS).

After Coolidge's attorney general, Harlan Fiske Stone, elevated Hoover to head the entire Bureau in 1924, he encouraged the new director to shift its focus from political surveillance to criminal law enforcement. But the FBI was soon back in the surveillance business. In September 1939, President Franklin Roosevelt signed a directive authorizing the FBI to investigate matters of espionage and sabotage and asked police departments to give the FBI all the information they collect about subversive activity.

FBI Director Hoover considered the directive to still be operational during the second Red Scare that followed World War II, and he greatly expanded the FBI's secret surveillance activity. In the early 1950's, the FBI and paid informants fed information to Sen. Joseph McCarthy as he carried out his televised hearings on “subversives” in the State Department and other agencies.

The FBI did not, however, have a monopoly on domestic surveillance. Other federal agencies eventually became involved in spying on the home front, among them the Central Intelligence Agency (CIA) – established by the National Security Act in 1947 to collect foreign intelligence and explicitly barred from exercising any “police, subpoena, law enforcement powers, or internal security functions” – the Internal Revenue Service (IRS), the Treasury's Secret Service, the Immigration and Naturalization Service (INS), the State Department's Passport Office, and the National Security Agency (NSA). The secret NSA had been created by President Harry Truman in 1952 to intercept communications from the USSR, and soon began monitoring the communications of Americans. In 1956, local and state police formed the Law Enforcement Intelligence Unit, which had its own Interstate Organized Crime Index of “terrorist” individuals and groups using the computer technology of the day, and its own Red Squad intelligence units, trained by the CIA. Police departments undertaking intelligence work sometimes worked closely with the FBI, foreshadowing the Joint Terrorism Task Forces of the 21st century.

The FBI's COINTELPRO, CIA's Operation CHAOS, and NSA's Operation SHAMROCK were some of the programs established to monitor and disrupt lawful First Amendment activity during the cold war period. In the name of keeping the country safe, infiltration, dirty tricks, psychological warfare and violence were used against political dissent, the movements for civil rights and black liberation, and protests against the Vietnam War, among other perceived threats.

Once the extent of government overreaching and abuse of power came to light through the post-Watergate investigations of the Rockefeller Commission and the Senate's Church Committee, the effort to regulate domestic spying produced the Foreign Intelligence Surveillance Act (FISA) of 1978, under which a secret court was set up to process requests for warrants for the physical and electronic collection of “foreign intelligence information” and spying on terrorist suspects within the United States. The Privacy Act of 1974 gave citizens the right to see some of the records that were held about them and to correct inaccuracies. But in the same period, the elements of a surveillance society were being assembled, as swiftly evolving computer technology gave birth to new forms of monitoring, data sharing and storage, with far-reaching implications for maintaining social control.

By the time the Soviet Union disintegrated in 1990 and 1991 and the war on Communism had been brought to a triumphant close, the intelligence bureaucracy was gearing up to engage with a new enemy, both at home and abroad. The FBI had been given primary responsibility for preventing and investigating acts of terrorism in 1986, and within a decade had created a Counterterrorism Center at FBI headquarters which was supposed to facilitate an exchange of information with the CIA, INS and other government agencies.

Who were the prime targets? Boston University law professor Susan Akram and University of California professor Kevin Johnson have described the demonization of Arabs and Muslims as terrorists in the closing decades of the 20th century in the effort to silence critics of US Middle East policy. The Congressional response to Timothy McVeigh's 1995 bombing of the Alfred P. Murrah Federal Building in Oklahoma City was the passage of the Antiterrorism and Effective Death Penalty Act, whose draconian provisions to detain on secret evidence and deport “alien terrorists” were used exclusively against people of Middle Eastern descent.

“He who is not with us, absolutely and without reserve of any kind, is against us, and should be treated as an enemy alien,” former president Theodore Roosevelt told the nation as the first Red Scare was getting underway.[1] In the wake of the attacks of September 11, 2001, this mindset commanded a global stage. “Either you are with us, or you are with the terrorists,” Bush warned the nations of the world in his September 20, 2001, address before the joint houses of Congress.

As we shall describe in the next installment, the launch of the 21st century war on terror accelerated the movement toward “Top Secret America” without fully addressing the failings of the intelligence bureaucracy that paved the way for the September 11 attacks.

1. Quoted in David M. Oshinsky, “A Conspiracy So Immense: The World of Joe McCarthy” (1983), p. 88