A Critical Inquiry on Data Visualization Based Interactive Documentary: “The Fallen of World War II” as an example

Prologue

When I first watched and interacted with The Fallen of the World War II data visualization based interactive documentary, I immediately wanted to share it with my daughter. She was at the high school at that time and had problems with the pedagogical form of the history class, which was based on memorizing some facts about historical events and figures in a chronological order with cause-effect chain. Though I have always insisted that history is an amazing discipline and the pedagogical form of the class is not just proper for both her and history, she was just smiling at me without any other response. I called and wanted her to have a look at The Fallen. While she was watching the documentary, I was watching her and trying to understand her reactions. I told her that there are two interactive intervals, in which she could interrogate the data sets. She navigated in the charts and looked at the data sets with questioning eyes. Later we discussed on the documentary and the history class. She confirmed that history class with data visualization based interactive documentaries such as The Fallen would be amazing (but she was hopeless it to happen)...

Introduction

In this paper, first, I will put the major differences of The Fallen as a data visualization based interactive documentary from a conventional documentary on historical facts. Here, the major differences not only emerge from its form of justifying the fundamental tendencies of being a documentary, but also because of its particular mode of production. Secondly, I will emphasize the dramatic power of The Fallen on its viewer. In this context, I will discuss respectively the “truth affect of scientific numbers” in The Fallen and the resonance between audiovisual elements (especially archival photograph use and sound design including musical score) and data visualization to create dramatic intervals. In short, here, I will make an attempt to define the regime of truth claim in The Fallen. Finally, I will focus on the interactive intervals of The Fallen. These intervals operate both as a pause in the film and as a stimulation for the viewer to become an active agent for interrogating the provided data. In this last part, I will explore the viewer engagement form of The Fallen and make an emphasis on its power of creating will to knowledge through interrogating data.

The major differences of The Fallen as a data visualization based interactive documentary from a conventional documentary on historical facts

In his renowned edited book Theorizing Documentary, Michael Renov states and elaborates “four fundamental tendencies of documentary.” These four fundamental tendencies are as follows: “1. To record, reveal, or preserve 2. To persuade or promote 3. To analyze or interrogate 4. To express” (Renov 1993, 21). Hence, I will use these four fundamental tendencies to clarify the differences of The Fallen as a data visualization based interactive documentary from a conventional documentary on historical facts. When we look at The Fallen, we realize that it fulfills all these fundamental tendencies. However, in The Fallen, these four tendencies operate within a different attitude in comparison to a conventional documentary film based on historical facts.

Starting with the first fundamental tendency, this interactive documentary uses the registered official data and archival photographs as the ‘records’ that constitute its presentation. Though any camera record has not been used in The Fallen, the film has a very cinematographic language. This has become possible by the design of data visualization and insert of archival photographs within a cinematographic regime. Therefore, official data in the form of animated charts have become ‘objective’ records with the support of inserted archival photographs that reveal different perspectives and affects. Neil Halloran, as the director of the film, has designed each frame with the virtual camera of animated documentary and reproduced the records with mathematical interpolation of data sets by coding. The sound design, which consists of audio effects, musical score and interpretive voice-over, operates also in a cinematographic form and envelops the visually recreated records. Therefore, we can say that, The Fallen is not produced by shooting the real life and reproducing records by a camera, but by designing each frame and recreating the data and reorganizing the archival photographic records by the virtual camera of animation with a cinematographic visualization and audio design.

Secondly, The Fallen persuades affectively its audience to reconsider peace and war with the account of the dramatic number of loss of soldiers and civilians in the Second World War. This persuasion seems as an intervention from which the viewer cannot avoid. The exactness and seriousness of mathematical calculation and statistical demonstration of the losses that is created by data visualization and evidentiary role of the inserted archival photographs make the film so persuasive. In other words, the abstract images created in the mind of the viewer by mathematical calculation and statistical demonstration are completed with a visual encounter with the archival images of the war. This persuasive power is also increased by the sound design and musical score, which both envelops the visual storyline and increases the dramatic affect of the film. Of course, the interpretive voice-over also has a significant persuasive power, as it has always been in conventional documentaries on historical facts. However, the persuasive power of The Fallen arises mostly from its visual channel that consists of the scientific justification power of mathematics and statistics. The audio channel operates more as a supportive enveloping element in the persuasion process.

Third, the strongest tendency of The Fallen may be stated as the “analysis and interrogation capacity” of it. Here, the basic difference of The Fallen in particular—and data visualization based interactive documentaries in general—from a conventional documentary film on historical facts is the changing analytical position of the viewer. In conventional documentary film, the viewer passively watches the analysis and is subject to interrogations made in/by the documentary on historical facts. In data visualization based interactive documentary, the viewer is stimulated to take an active position and make the interrogation herself through analyzing the data sets on historical facts. Mathematical analysis of the data and statistical representations screened (out of the interactive intervals) empower the viewer by providing models of analysis that she can make. Hence, these mathematical calculation and statistical representation models enforce the viewer for taking an active interrogative position. She is expected to formulate new perspectives by formulating new questions. Therefore, the provided data is subjected to the viewer in data visualization based interactive documentary, rather than she has been subjected to the analysis and interrogations made in/by the conventional documentaries on historical facts.

Finally, The Fallen reproduces a different mode of expression by using data visualization and archival photographs that are enveloped with sound design, musical score and interpretive voice-over. This expression utilizes analytical thinking through mathematical calculation and statistical representation. It emphasizes the significance of numbers with powerful affects of archival photographs, sound design, musical score, and interpretive voice-over. By the assemblage of all these elements within its cinematographic “montage design,” The Fallen eroticizes data, stimulates emotions, and creates desire to know, understand, and make an account of war and peace through life itself. Therefore, The Fallen’s viewer has a desire to know within its mode of expression. This is an unavoidable desire mechanism created by and provided to the viewer in the pedagogical interventionist expression of the film. In the establishment sequence of The Fallen, Neil Halloran, as the director of the film, presents the basic mathematical postulates, simple counting methods, and affective statistical representations of the data to the viewer. Hence, the viewer learns how she can follow the film. In the following sequences of the war and peace narrative, this time, the viewer gets how she can make similar analysis and interrogation of the data. After watching the narrative sequences that are established on the interrogative analysis of the data, in the interactive intervals, the data is openly provided to the viewer in which she can make similar analysis and interrogation. Therefore, with the capability gained from the pedagogical interventionist expression of the film, the viewer falls into “libido sciendi” and formulates questions or interprets the data in an interrogative analytic manner.¹

Dramatic power of The Fallen on its viewer

In this section, I will discuss respectively the “truth affect of scientific numbers” in The Fallen and the resonance between audiovisual elements (especially archival photograph use and sound design including musical score) and data visualization to create dramatic intervals. In short, here, I will make an attempt to define the regime of truth claim in The Fallen.

Today, in the age of (big) data, the business world, public policy making and academic circles are all in the effort of attaching and legitimizing their positions and assertions with data by the operationalization of business analytics and/or data science. In a sense, today, truth can be more legitimately claimed by data and clearly represented by its visualization. In this vein, documentary filmmaking has always utilized data for its truth claim. But the rules of the game changed with the Internet. The Internet became the primary medium of continuous exponential generation of data. Hence, the interactive documentary, which is natively Internet based, has sought new modes that are based on data and its visualization. In parallel to that, there appeared a change in the regime of truth claim in data visualization based interactive documentary.

Data visualization based interactive documentary is relatively a new mode in documentary filmmaking. Though we know conventional regimes of truth claim in documentary, we have to explore the regime of truth claim in data visualization based interactive documentary. Here, I will discuss the regime of truth claim of The Fallen as an example. It is so obvious that mathematical calculations and statistical representations play an important role in the regime of truth claim of The Fallen. We may call this the truth affect of scientific numbers and their scientific representation. In addition to that, the regime of truth claim in The Fallen is reinforced by the cinematographic interweaving of data visualization with audiovisual elements. First, I will discuss the truth affect of scientific numbers and their scientific representation.

The truth affect of scientific numbers and representation in The Fallen

The scientific observation goes back to ancient Greek culture. In the beginning, it was simply observing with senses by being (there) in the observation environment, and taking down notes and drawing illustrations as Aristotle did as a naturalist. He made observations on natural life and documented them in his field notebooks as notes and drawings.² Later, scientific apparatuses such as thermometer, barometer, telescope, microscope, etc were invented. These apparatuses not only made scientific observations more precise and accurate but scientifically justified the observation itself. Later camera came and gained the status of being a scientific apparatus within its invention. According to Brian Winston, this has two main reasons. First, pictorial representation has become “a major mode of scientific evidence” within its history which in turn triggered and partially caused the invention of the camera itself. Secondly, there has always been a tendency to utilize “instruments of inscription” to produce data. In that sense, camera has been a powerful instrument of inscription for many scientific disciplines (Winston 1993, 37).

According to Winston, “modern science depends on modes of representations” that are reproduced by instruments of inscription (1993, 40). In his elaboration, Winston takes sociologist Bruno Latour’s social character “obstinate dissenter,” who does not believe the scientific outputs and is insistent to interrogate the scientist. The obstinate dissenter questions the formulation and process of scientific experiment. When this social character is permitted to enter into the laboratory to question the process there, she witnesses the reproduction of a shape by a device as a result of the experiment. The obstinate dissenter makes a significant observation at his point as Latour puts it: “We are at the junction of two worlds: a paper world that we have just left, and one of instruments we are just entering. A hybrid is produced at the interface: a raw image, to be used later in an article, that is emerging from an instrument” (quoted by Winston 1993, 41).

Winston concludes that “Latour equates scientific instrument with inscription device” (Winston 1993, 41). According to Latour, scientific texts are written based on the inscriptions that are reproduced by scientific instruments. In the setup of scientific instruments, both the scientist and the obstinate dissenter attend an “audio-visual” spectacle. As a result, the scientific knowledge is an output of “a visual set of inscriptions produced by the instrument and a verbal commentary uttered by the scientist” (Winston 1993, 41). At this point, Winston emphasizes that the position of the obstinate dissenter is very similar to the position of the viewer of a documentary film.

When documentary filmmakers liken their work to data collection or “voyages of discovery,” they implicitly position their audiences as Latourian obstinate dissenters who have penetrated the lab of their filmmaking experiments. All the filmmakers’ off-screen denials of objectivity, all off-screen protestations as to their own subjectivity (should they make them), are contained and, indeed, contradicted by the overwhelmingly powerful cultural context of science. (…) The centrality of this scientific connection to documentary is the most potent (and sole) legitimation for its evidentiary pretensions. Thus, documentarists cannot readily avoid the scientific and evidential because those contexts are “built in” to the cinematographic apparatus (Winston 1993, 41).

Here, we should open Winston’s statement: When he states that the scientific and evidential contexts are “built in” to the cinematographic apparatus, he not only means the camera by itself, but the whole cinematographic apparatus including the camera as well. Here, the cinematographic apparatus consists of the cinematic narrative, the production process, the montage regime with all its audio-visual elements, techniques, and discourse. Therefore, Neil Halloran’s data visualization based interactive documentary The Fallen, in which no real camera is used, still has a built in scientific and evidential context with its cinematographic regime. Here I will discuss how the truth claim of The Fallen operates without an evidentiary picture that is reproduced by a camera?

Indeed, two different elements establish the built in scientific and evidential context of The Fallen. First, the data set consisting of numbers of dead soldiers and civilians in the Second World War—of which its source is given as a reference at the credits of the film—maintain basically the scientific and evidential context. Here, the common public opinion about the scientific truthness of statistical data sets operates as the main justifier of the evidential context. However, the viewer mostly does not make an inquiry on the scientific relevance of the statistical data sets. Secondly, the statistical representation of the data sets by coding and interface design in a cinematographic montage regime both maintain and reinforce the scientific and evidential contexts. Here, the film utilizes not only the statistical representation but scientific language in its interpretive voice-over as well. At this point, for being able to reveal the role of the audio elements in the regime of truth claim of The Fallen, it becomes necessary to elaborate the sound design of the film.

The resonance between cinematographic audiovisual elements and data visualization

Sound design of The Fallen consists of voice-over, sound design (of ambient sounds and sound effects), and musical score. These sound elements are designed and composed by Andy Dollerson. First I will explore each of these elements separately. Then I will discuss their combination in themselves and in relation with the visual channel.

The voice-over of The Fallen is based on a well scripted narrative, which expands the affect of the visual channel of the film. As a common critique on voice-over in documentary cinema, this voice-over can also be called as “voice-of-God” commentary with its theatrically toned, well trained, authoritative male voice coming out of the frame. However, this time, it operates more in an interpretive manner with the sake of “thinly masking the impending chaos underlying documentary film’s organization” (Takahashi 2017, 383). Its main intention can be defined as not only guiding the viewer for interpreting the numbers but encouraging her to learn to interpret them as well. In the design of the voice-over, dramatic gaps are established in which the voice-over goes into silence. In these gaps (such as the number of dead American soldiers (1:45-1:55); the dramatic number of dead Soviet soldiers (06:10-06:47); the number of Jewish people killed in the Holocaust (07:35-07:44)) data metaphorically “speaks for itself” (Takahashi 2017, 383). Although there is no voice-over in these dramatic gaps, there are ambient sounds such as wind and dramatic musical score. Takashi calls these sound elements “murmur.” According to Takashi a murmur does not necessarily reference a human voice and can be produced by any technological means or be emitted by elements of the natural world such as wind, trees, etc. or the soundless sound of outer space. Takashi states that “whether human, machinic, natural, or animal, the murmur is figured as more of a continuous vibration than a sound object” and emphasizes that “the murmur thus exceeds the human as a speaking category, important for a time in which documentary is called on to speak” (Takahashi 2017, 384). Especially, the ticking sound in combination with the wind sound, as murmurs, in the rising numbers of dead Soviet soldiers scene really creates a very dramatic interval. Andy Dollerson’s sound design, with its well designed interrelation of sound elements, makes the data not only be understood but deeply felt by the viewer as well.

In sum, we can say that, the regime of truth claim of The Fallen operates as a cumulative of the truthness of the scientific numbers with their statistical representation and the resonance between the data visualization and sound design, all within a cinematographic montage regime. In this montage regime, two interactive intervals come to fore and break the rules of the game. Not only the voice-over goes into silence but, except the murmurs, the film totally gives a pause to its audio-visual elements in its flow as well. Now the viewer becomes the director and takes the control of the screen.

Interactive intervals of The Fallen

The film includes two interactive intervals. These intervals operate both as a pause in the flow of the film and as a stimulation for the viewer to become an active agent for interrogating the provided data. In this last part, I will explore the viewer engagement form of The Fallen and make an emphasis on its power of creating will to reproduce knowledge through interrogating data.

Lev Manovich states that “along with a graphical user interface, a database, navigable space, and simulation, dynamic data visualization is one of the genuinely new cultural forms enabled by computing” (Manovich 2008, 3). With this introductory statement, Manovich emphasizes two important aspects of computer based “dynamic” data visualization of our time, which is animated and/or interactive rather than static images. First, he lists the computational elements which make computer-based “dynamic” data visualization possible as follows: development of graphical user interfaces for various computer operating systems; availability of different types of databases for different computational purposes; design of navigable spaces in which user can go into interaction by making selections among options; and capability of representing data through the simulation of mathematical functions. Secondly, he underlines the affordances of the computer medium that makes the dynamic data visualization widespread and “normal.” With the affordances of today’s computers, not only larger data sets can be visualized but also interactivity can be maintained within the data provided. Manovich calls this capacity “reversibility” (Manovich 2008, 6). Here, reversibility emerges from the capacity of interacting with the visualization. In a reversible interactive data visualization project such as The Fallen, the viewer can get more information that reproduced the statistical representations. Hence, the viewer gains the capability of interrogating the provided data and brings new perspectives by formulating new questions.

Conclusion

Today, data visualization based interactive documentary is iconically pointing out the coming future of truth claim of documentary. This new mode of documentary makes big data, which is beyond the scale of human senses, more conceivable, visible and tangible (Manovich 2008, 7). Within the limits of its interactive capabilities, it encourages its viewer to interrogate the provided data. In The Fallen, the flow of the narrative is paused twice and while the screen is turned into an interface, the viewer becomes a “user-player.” Indeed, the non-interactive flow of the narrative operates as the pedagogical empowerment of the viewer in learning how to interrogate data and make an alternative analysis on it.

However, there are also some critical issues we can discuss. Data visualization based interactive documentary (The Fallen as the object of inquiry in this paper) is designed mostly on the presumption of the existence of “statistically literate viewer.” Although the narrative of the voice-over is simple in The Fallen, the film still presumes that the viewer has the basic notions of statistical representation. Of course, no data visualization based interactive documentary is expected to be proper for everyone. What I mean is that, we have to realize the literacy is a significant issue when we are discussing data visualization based interactive documentary.

The Fallen makes a critical inquiry of the Second World War through the number of losses, i.e. life itself. However, absolutely numeric representation takes the discussion towards an economy of life and out of a political context in a sense. Though the aim of it is to expose the dramatic number of losses from different nationalities at a great geographical scope, the political context of discussion on war and peace should never be forgotten...

Final Notes

¹The Latin term “libido sciendi” is inspirationally explained by Foucault in an interview made with him. You can access this interview with Spanish subtitles via the link <https://www.youtube.com/watch?v=_rYmMTIZPHM&t=200s> and with Turkish subtitles via the link <https://www.youtube.com/watch?v=PxJaP-7dQbY>.

²For Aristotles’s observations as a naturalist you may look at https://www.nature.com/articles/091201a0 “Aristotle as a Naturalist.” Nature 91, 201–204 (1913). https://doi.org/10.1038/091201a0.

Bibliography

Manovich, Lev. 2008. “Data Visualization as New Abstraction and as Anti-Sublime.” In Small Tech: The Culture of Digital Tools, edited by Byron Hawk, David M. Rieder, and Olliee Oviedod, 3–9. Minnesota: University of Minnesota Press.

Renov, Michael. 1993. “Towards a Poetics of Documentary.” In Theorizing Documentary, edited by Michale Renov, 12–36. London: Routledge.

Takahashi, Tess. 2017. “Data Visualization as Documentary Form: The Murmur of Digital Magnitude.” in Discourse Vol. 39, No. 3, 376-396.

Winston, Brian. 1993. “The Documentary Film as Scientific Inscription.” In Theorizing Documentary, edited by Michale Renov, 37–57. London: Routledge.