Current events

Text vs Visual. Multimodal challenges in the Digital Humanities

A multi-part event at the Centre for Information Modelling at the University of Graz.

Scholarly knowledge is created through the interaction of different modalities. The written and printed word traditionally occupies a central position. In contrast, visuali-sations have established themselves as an important alternative in the scientific com-munity. They have evolved from a purely representational element in the communica-tion of already formulated scientific results to a mostly interested lay audience to an important methodological approach in the development of knowledge.
Until now, these two modes have been largely considered separately from each other and from a single disciplinary perspective. Texts and visualisations are seen as complementary ways of creating and communicating knowledge. The question of the direct relationship between these two modes remains largely unresolved. Who as-sumes which functions in the constitution of scholarly knowledge? What alternative approaches exist beyond the opposition of linear forms of appearance (text) and spatial arrangements (visualisations)? Which hybrid and overlapping solutions can be used effectively?
Different perspectives on these questions will be discussed in this event format. Aspects of both digital and traditional humanities will be addressed and practical ap-proaches will be emphasised. The format consists of several parts, at the end of which a position paper will be published. Interested parties will have the opportunity to ac-tively participate in this process through a keynote speech, a hybrid workshop and a public review process.

The program:

Wednesday, May 8 2024, 12:00-1:00 p.m
keynote by Stefan Haas (Göttingen) “Visualization as Gamechanger in Historiography”
Venue: Zentrum für Informationsmodellierung, Elisabethstraße 59/III, 8010 Graz, 3rd floor, room 81.31
Link to uniMEET
Abstract: Visualization is one of the most consequential game changers in all contemporary sciences. It changes how we encode knowledge in a medium and how we decode it in the process of reception. By doing so the structure of knowledge itself is changed. This is particu-larly evident in two areas: heuristics, in which hypotheses are derived from a first sketch of the empirical material, and narration, in which knowledge is passed on. The lecture deals with the basic structures of this fundamental change.

Tuesday, May 14 2024, 5:00-6:30 p.m
round table “Text vs. Visual. Methodological approaches and practical applications” with Linda Freyberg (Berlin), Philipp Hofeneder (Graz), Christian Wachter (Bielefeld), Florian Windhager (Krems)
Venue: Zentrum für Informationsmodellierung, Elisabethstraße 59/III, 8010 Graz, 3rd floor, room 81.31
Link to uniMEET
Abstract: This round table will discuss the relationship between text and visualization from different perspectives. Four different experts will present their perspectives in ten minutes. Afterwards, there will be an opportunity to ask questions and make comments. A detailed presentation of the individual contributions can be found below.

Wednesday, May 15 2024, 9:00 a.m. - 3:00 p.m
closed workshop with the participants of the round table working on a joint paper

September – December 2024
public peer review of joint paper at the “Zeitschrift für digitale Geisteswissenschaften”; reviews by Jana Pflaeging (Salzburg), Tuomo Hiippala (Helsinki)

2025
publication of joint paper

Abstracts for the round table on May 14:

Linda FREYBERG
Image vs. Text - Visualizations as epistemic agents for the Digital Humanities

From a semiotic point of view, visualizations firstly are images and act as icon signs, which in terms of diagrammatic reasoning have an epistemic effect. According to Peirce “[a]ll necessary reasoning without exception is diagrammatic“. In particular, based on abductive reasoning visualizations are able to create new knowledge on account of their topological depiction of objects and their relations. In comparison to that textual expressions are strictly symbolical signs with (in most languages) an arbitrary relation to their represented objects. On the one hand text is highly restricted by the rules of a language and is limited in terms of creative expression within the reasoning process. On the other hand, text can be highly precise and accurate. Both forms of expression are embedded in a specific cultural and historical context. Therefore, visualizations also act as symbols and serve as iconic agents through their iconographical embodiment, which reflect on, but also contribute to cultural change. Just like images, visualizations may suggest evidence or realities that remain vague or suggestive only. In Digital Humanities research in particular the realm of cultural heritage data calls for both, accuracy and plurality in order to express explorative and narrative approaches to multimodal data.

Philipp HOFENEDER
Visualization Data Sheets - providing sources for academic reusability

The aim of scholarly work is to generate organised and reliable knowledge. To this end, primary and secondary sources are cited as well as direct and indirect quotations that categorise one's own findings in existing research and make them comprehensible. While these elements are among the basic requirements for texts, comparable approaches for visualisations are only in their infancy. In this contribution, I would like to discuss data sheets as a possible approach for visualisations in a scientific context. These data sheets are known from industry (e.g. for components, materials, products) and describe the properties, possible applications and technical data of. Data sheets for visualisations could include technical questions (which programs were used, where and in what form should the visualisation be used?), conceptual considerations (who is the visualisation aimed at, what should it achieve?), possible templates (best practice examples) and the data on which the visualisation is based. As such they not only provide a compact overview of relevant information on a visualisation and thus significantly increase usability, but also make visualisations - similar to texts - easier to understand and cite.

Christian WACHTER
Mapping Knowledge: Navigating the Complex Web of Ideas in Scholarly Publications through Multimodal Digital Interfaces

Humanities scholars structure their works according to the logical progression of their arguments and narrative order. This construction is mirrored in the layout of chapters and sections, where the table of contents serves as a symbolic tool that is, in fact, multimodal because it spatially organizes these elements. Although this method adeptly represents the linear succession of argumentative and narrative steps, it cannot fully capture multivocal concepts—historical entanglements, diverse viewpoints in societal discourses, and other nonlinear phenomena that defy straightforward hierarchical categorization. In this light, my talk proposes expanding the multimodal capacities of tables of contents in digital publications through interactive abstract maps, akin to concept maps. While there are existing efforts to merge traditional content tables with dynamic maps, they often fall short, either clinging too tightly to conventional formats or creating abstract networks that lack clear, user-friendly labels. My discussion will pivot on strategies for developing content maps that reveal more explicitly the publication’s contents and their flexible interconnections. Exploiting the multimodal potential of textual and visual media is fundamental for this goal.

Florian WINDHAGER
Worth 10.000 Words? - You wish!
On Semiotic Modality Wars & (Future) Collaborations in Humanities Departments

It’s simple, technically: We have to stake out and stabilize a position against the iconoclasts (i.e. humanists, mindlessly thrashing visualizations), but also against the logoclasts (i.e. visualization and computer scientists, not aware of the relevance and richness of humanists’ text-based discourse). Obviously, getting there would be easier, if we would have benefitted from a more multimodal and collegial understanding of both the world and our (commonly quite multimodal) thinking about it. Or put in structural terms: From a more balanced training of textual and visual literacies—and from a more even-keeled culture of scholarly knowledge representation. That would help to be aware of the obvious strengths-and-limitations of both (and other) semiotic modalities and all their instantiations. And thus also to know about the stunning benefits of combining them for deeper sensemaking within the semiotic torrents and flash floods of modern society. So that we can cut short on the -clasms and move towards closer collaborations on more productive meshworks of hyper-image-&-text-assemblies. Which are arguably needed everywhere. The overall aim is to couch this conflictoclastic message and the related multi-century working-programme in an agreeable and reasonably short statement.