He was elected a corporate vice president in 1966, appointed to the management committee in 1967, promoted to vice president of corporate finance and planning in 1968, and promoted to senior vice president in 1969. In 1974, Opel was elected president of IBM. On January 1, 1981, he became IBM's CEO and chairman of the management committee.
Opel served as a trustee for Westminster College, the Institute for Advanced Study, the Economic Club of New York, and the Joint Council on Economic Education. He was a director of the Bank of New York, the Bank of New York Co., Inc., and Pfizer, Inc. He also served on the board of governors of United Way of America and was a member of the Council of the Woodrow Wilson International Center for Scholars.