Lillian Schwartz & the Mona Leo Theory

Lillian Schwartz first met computer scientist Gerard J. Holzmann at Bell Laboratories, where she had held a (mostly unofficial) position as a “resident artist” since 1969. Holzmann was embedded in the Computing Sciences Research Center—the origin point for the UNIX operating system. Similar to many other divisions at the Labs, Holzmann later described the CSRC as “a truly remarkable group that was given virtually complete freedom to pursue research goals, with no perceptible management oversight.

PICO portrait of Lillian Schwartz detailing transition from matte images into a composite with digital mosaic background. Images by Gerard J. Holzmann, Beyond Photography: The Digital Darkroom, page 70-71.

It was in this atmosphere of free exploration that Holzmann and his colleagues Rob Pike and Ken Thompson worked out a new type of digital image processing program. The program was named “PICO” and they likened it to a “digital darkroom.” PICO was created in 1984, six years before Photoshop 1.0, but similarly, it could digitally scale, rotate, mirror, merge, and juxtapose images—without a drop of darkroom chemistry in sight. Today, these types of adjustments are easily handled by our smartphone apps. But at the time of PICO’s creation, high resolution digital image manipulation required technical skill, as is evident in Holzmann’s 1988 book, Beyond Photography.

Backtracking to 1982, Schwartz had been commissioned by the Museum of Modern Art and given funding by IBM to create a computer-generated poster and short animation to announce the re-opening of MoMA’s new galleries. This project expanded Lillian’s circle of cross-disciplinary collaboration beyond Bell Labs and into IBM’s Thomas J. Watson Research Center in Yorktown Heights, New York. As part of her ideation, Schwartz worked closely with IBM physicist Richard F. Voss, an expert in fractal geometry. She experimented with real-time interactive computers (as opposed to her 1970s work with punch-card processing) to create digital renderings of MoMA's galleries, describing in her journal "strange perspectives, impossible without computer[s].” Meanwhile, Voss worked on fine-tuning computer programs to support her artistic vision. He created a digital mapping tool that allowed Lillian to “hang paintings” in digital space, drawn from scans of MoMA’s signature art collection.

Lillian Schwartz at work on the MoMA commission at IBM Thomas J. Watson Research Center, 1982-1983 (left). / THF537660; Digital rendering of MoMA gallery by Schwartz with programming by Richard F. Voss (right). / THF706851 

In an August 21, 1982 diary entry, Schwartz wrote: “Great adventure at this stage of using computer. Feeling of strength, power in ‘building’ tools to create with; searching for elements essential to ‘beautiful objects.’ Must have flexibility. Not be locked in by programs.” She went well beyond the terms of her MoMA contract—seemingly to the annoyance of the commissioning committee. She created digital palettes, which led to a series of fractal, crystalline “remixes” of new art formed from the museum’s collection.

Digital art by Lillian Schwartz using Richard F. Voss’s programs: “Big MoMA,” 1984 (left). / THF196222; A computer-generated fractal cat, 1983 (right). Look closely—the eyes are from René Magritte's "False Mirror" painting. / THF705879 

These works document Schwartz’s deep fascination with the technical and perceptual processes of historical artists. In the act of thinking through art history—by channeling “the masters” through the filters of new technologies as data points—Schwartz surfaced creative work that was not outright appropriation. This mode of visual computing hints at “the generative”—nearly half a century before we could enter the now-familiar command “create a portrait in the style of [insert your favorite artist name here]” into AI-image generators like DALL-E or Midjourney. In another entry, she wrote:

It was this line of questioning—digitally scanning, analyzing, and manipulating historical artwork—that led Lillian to Gerard J. Holzmann’s lab one day. After completing her MoMA contract, she was back at Bell Labs more regularly. In the mid-1970s, she had prepared a “morph” image for a lecture at Rockefeller University to compare similarities in the facial bone structures of historic figures. To demonstrate, she transformed a portrait of Rembrandt into Einstein, and then back again, in nine steps. Her intention was to apply this process to Leonardo da Vinci’s work, but she moved on to other projects. Schwartz returned to the concept sometime in 1984, but technology had progressed, and her magtape files had degraded. Holzmann invited her to use his equipment to make new scans, and to try out an early version of his PICO program. 

Lillian Schwartz, “Rembrandt to Einstein,” 1975. Appears in The Computer Artist’s Handbook, by Lillian F. Schwartz with Laurens R. Schwartz, page 271 (WW Norton & Co Inc, 1992) .

Schwartz’s interest in Leonardo da Vinci (especially the Mona Lisa) began many decades prior. In a fax to her publisher, she shared her first experience with the painting: “1963: The Met. Finally saw the Mona Lisa or did I. Traffic jams. Huge crowds. […] Then there it was. Necks craning. So small. The lighting was bad. Glares. Too much varnish? People quiet, staring, trying to suck in Mona. What is it that makes it so unique? I wasn’t sure I even liked it.” But then, a later entry, perhaps the next day: “Couldn’t sleep. It’s grabbed me. I’m hooked. Still not sure whether it was the reactions of the people to Mona or mine. Strange painting.” She wrote: “By that time, I was haunted by the Mona Lisa.”  

With Holzmann’s assistance, she re-scanned da Vinci’s Portrait of a Man in Red Chalk (believed to be a self-portrait), and the Mona Lisa. With these resources in hand, Schwartz was ready to interrogate da Vinci’s artwork. The portraits became datasets. Facial structures were treated as topographical maps. The eyes became markers for alignment. Since PICO did not operate in a “point and click” graphic user interface environment, Schwartz learned to run UNIX scripts to mirror, scale, and then join the two images together into a composite.

The spliced image that emerged line-by-line on the computer monitor revealed an “atypically perfect” match that Schwartz described as “exhilarating.” At the center line, the lips, eyes, and mouths of each image lined up within a 2% margin. Did this prove that da Vinci himself served as the model to complete the painting? Holzmann didn’t think so, and even Schwartz was initially cautious, recounting: “…this was an instance where the computer suggested a conclusion that it had not been asked to make. However exciting, the unexpected result had to be confirmed.” She retreated into research, only to run up against the same documentation gaps of the Mona Lisa model’s identity that had flummoxed many art historians in the past.  

Lillian Schwartz, “Mona Leo” digital print, 1988 (left). / THF268991; Composite of Mona / Leo with parallel lines establishing continuity of facial features. Appears in The Computer Artist’s Handbook, by Lillian F. Schwartz with Laurens R. Schwartz (WW Norton & Co Inc, 1992), page 274 (right).

Illustration from “Morphing the Three Faces of Mona: The Decision-Making Steps Leonardo Used to Create His Mona Lisa,” article by Lillian F. Schwartz in Computers & Graphics, Vol. 19, No. 4, 1995, page 335. 

Ultimately, Schwartz settled into the belief that digital analysis had unlocked proof that the Mona Lisa was in fact a partial self-portrait of da Vinci. Her theory made headlines in major publications and academic journals, first appearing as a cover story in the January 1987 issue of Art & Antiques. Additional coverage followed in The Visual Computer (1988), Omni (1990), Scientific American (1995), and many others. She was interviewed on CBS Evening News, and a reenactment of her discovery was featured on a 1993 episode of Unsolved Mysteries.  

A selection of publications featuring coverage of Schwartz’s “Mona Leo” investigations: Art & Antiques, January 1987 (left, unprocessed collection); Omni, October 1990 (center). / THF706097; Scientific American, April 1995 (right). / THF706084

Still images from the “Mona Lisa” segment of Unsolved Mysteries, Season 6, Episode 9, Cosgrove-Meurer Productions / FilmRise, 1993-1994. The real Lillian Schwartz interviewed (left), and a reenactment of her discovery played by actors (right).  

Schwartz’s “Mona Leo” concept was polarizing. Science and computing fields embraced her data-driven methodologies to the point where her work appeared in peer-reviewed journals. The media fed into the spectacle of the mystery. However, art and humanities scholars were skeptical of the “male Mona Lisa.” Despite many unknowns in the historical record, it is generally believed that the model for the Mona Lisa was Italian noblewoman Lisa del Giocondo. Throughout the late-1980s, Schwartz continued her digital analysis through projects such as “unmasking” the identity of the model for an engraving of William Shakespeare, authenticating a photograph of Emily Dickenson, and reconstructing the perspective for da Vinci’s “Last Supper.” 

Examples of press coverage of Schwartz’s digital analysis of the engraved portrait published in William Shakespeare’s “first folio.” Schwartz posited that Queen Elizabeth I served as the model, with masculine features added. The Star-Ledger, 23 March 1992 (left). / unprocessed collection; Clipping from unknown publication, 1992 (right). / THF706093 

Putting aside the tensions of proof and skepticism, Schwartz was a noteworthy advocate who positioned the computer as a toolkit capable of supporting creative work and scientific analysis in tandem. As a “computer detective,” she advanced the idea that a digital surrogate could hold more information and truth than an original. And as an artist, she brought the imaginative curiosity needed to interrogate the fluid complexities of time, truth, and identity. The “Mona-Leo” project can also be considered as a portal for the history of forensic visual computing—echoing modern biometrics and facial recognition technologies—along with their known failures and complications.  

In The Computer Artist’s Handbook, she wrote: “The computer can be used to study creativity, to compare art movements, to restore damaged frescoes—even to solve mysteries. […] The computer can give us time—the time to associate what we know for the sake of our artistic eye, and the time to analyze what we do not know, perhaps to the benefit of all humanity.” 

Unpublished autobiography by Lillian Schwartz, c. 1986

“The Mona Lisa identification: Evidence from a computer analysis,” draft manuscript, c. 1988

“The Mona Lisa identification: Evidence from a Computer Analysis,” The Visual Computer (1988), 4: 40-48, Springer-Verlag

“The Computer: A Detective Tool in the Humanities,” draft article outline faxed to editor at WW Norton, 1991

The Computer Artist’s Handbook, Lillian F. Schwartz with Laurens R. Schwartz, WW Norton, 1992

“The Identity of the Mona Lisa: Whose Smile Is It Anyway?” draft manuscript, c. 1993

“The Two Mona Lisas: Tales of a Cultural Detective,” draft manuscript, c. 1993