Perlis was born to a Jewish family in Pittsburgh, Pennsylvania. In 1943, he received his bachelor's degree in chemistry from the Carnegie Institute of Technology (now Carnegie Mellon University). During World War II, he served in the U.S. Army, where he became interested in mathematics. He then earned both a master's degree (1949) and a Ph.D. (1950) in mathematics at MIT. His doctoral dissertation was titled "On Integral Equations, Their Solution by Iteration and Analytic Continuation."

In 1952, he participated in Project Whirlwind. He joined the faculty at Purdue University and then moved to the Carnegie Institute of Technology in 1956. He was chair of mathematics and then the first head of the Computer Science Department. He was elected president of the Association for Computing Machinery in 1962.

He was awarded the Turing Award in 1966, according to the citation, for his influence in the area of advanced programming techniques and compiler construction. This is a reference to the work he had done as a member of the team that developed the ALGOL programming language.

In 1971, Perlis moved to Yale University to become the chair of Computer Science and hold the Eugene Higgins chair. Perlis was elected to the National Academy of Engineering in 1977.

In 1982, he wrote an article, Epigrams on Programming, for ACM's SIGPLAN journal, describing in one-sentence distillations many of the things he had learned about programming over his career. The epigrams have been widely quoted.

He remained at Yale until his death in 1990.