Computation & AI for Creativity on Nov 14th, 2022

I am organizing aRobot painting a picture seminar on Copenhagen this November 14th.

Do new computational and AI tools for generating poetry, concept art, stories, and visual art fundamentally *change* creativity, and do they allow for new kinds of art?

In this seminar, open to all, five artists and writers share their process, results, and ideas about using computation and AI for creative work.

Monday November 14th, 2022, 16:00-18:00
Room 90.1.25, Royal Danish Academy, Fabrikmestervej 10, 1435 Copenhagen K.

We have five amazing speakers, each of which uses computation and AI for creative ends:

  • Nick Montfort (MIT): “Computation and creativity in literature”
  • Charlene Putney (Writer, Laika) & Martin Pichlmair (ITU, Laika): “Creating Laika, an AI tool for story writers”
  • Ida Kvetny (Visual artist): “Using AI image generation as a visual artist”
  • Lukas Damgaard (Freelance visual developer): “Building worlds using Midjourney”

Organized by Jesper Juul / Visual Game & Media Design / Institute of Visual Design at the The Royal Danish Academy
Info: jjuul@kglakademi.dk

Speaker Bios

Nick Montfort is a poet and artist using computation as his main medium. Computer-generated books of his include #! and Golem. The MIT Press has published his Twisty Little Passages, The Future, and Exploratory Programming for the Arts and Humanities. He directs  a lab/studio, The Trope Tank, and is professor of digital media at MIT. He lives in New York City.

Ida Kvetny is an interdisciplinary artist based in Copenhagen, Denmark. Through VR, AR, NFT and AI Kvetny merges the digital with her works in paint and clay, and thus creates a multimodal visual world. Herein, the unconscious occupies a privileged space, where her intuitive approach to image creation leads to places unreachable by rationality.

Dr Martin Pichlmair is Associate Professor at the Creative AI Lab in ITU, and a veteran entrepreneur with several games studios under his belt including the multi-award-winning Broken Rules.
 
Charlene Putney is an award-winning writer and teacher from Ireland. After working at Google and Facebook in management positions, she’s been writing for videogames since 2013, including writing for Divinity: Original Sin 2 and Baldur’s Gate 3.
Together, Martin and Charlene are working on LAIKA, an AI-powered creativity tool for writers. You can sign up for the waitlist at www.writewithlaika.com.

Lukas Damgaard is a film animation director from the Danish Film School. He works as a freelance visual developer in the animation industry.

PhD Defense of Milan Jaćević: A Study in Practice

PhD Thesis coverJoin us for the PhD Defense of Milan Jaćević this Friday September 2nd, 2022 at 1 PM Copenhagen Time (UTC+2) in-person or on Zoom.

The dissertation is “A Study in Practice: The structure and functioning of Ludic Habitus in Interactions with Digital Games”.

Using game prototypes and player studies, the dissertation develops “a general framework of digital gaming practice, a theory of digital gaming that explains how humans develop into players over the course of multiple acts of play, and how these prior experiences help to structure their understanding and behavior in subsequent gaming situations.”

The assessment committee consists of Kristine Jørgensen (University of Bergen), Staffan Björk (University of Gothenburg), and chair Alessandro Canossa (Royal Danish Academy).

The thesis can be downloaded here.

The Meanings & Consequences of Rules & Algorithms

TL;DR; I used to meet resistance to the idea that video game rules (usually expressed in the programming) and algorithms can be meaningful or important. I discuss four ways game rules can create meaning and/or have political implications.

When I first started writing about video games, the humanities  (full disclosure: I am a humanities scholar) were probably at the peak of self-importance, and many of the theories we were taught were presented as timeless and universal for explaining all of human culture for all time. It wasn’t really on the table that these theories should be augmented with anything specific going on in, say, video games, or in anything else, new or old.

When I then excitedly tried to argue that the rules of video games could also be central to the game’s attraction and to their meaning, and should be included in the humanities, this came across as boring, nerdy tech stuff that many people couldn’t believe had any importance, and felt about as relevant to them as the construction of printer drivers.

September 12th game
September 12th

I realized the other day that it’s been years since I had this experience of someone dismissing game rules as irrelevant or meaningless, and I think there is a reason: Our world is now so completely enmeshed in algorithms and in issues of algorithmic bias, that it’s now a given that rules, algorithms, and programming fundamentally matter.

In many ways, of course, but I can see that I have thought mostly about four ways that rules matter.

1) Meaning and Political Expression

New and popular culture is often dismissed as “meaningless”, and video games have been dismissed this way along with (say) jazz and romance novels. How then to explain why we find video games meaningful?

A central early and influential example is Gonzalo Frasca’s game September 12th, which at first gives the impression of being a kill-the-terrorists-and-you’re-done game, but once you play, the game reveals how killing breeds resentment and perpetuates a cycle of violence. This made September 12th an influential early example of how the rule system of a game could be expressive, and here express a political point about the counter-productiveness of the War on Terror.

Obviously, most games combine some kind of fictional world with a rule-based system. In the September 12th case, the genre expectations set up by the fiction & visual representation is undercut by the rule system, and we are as players forced to reconsider the actions we are performing in-game. What we think of as meaning is usually a combination of rules, fiction, but also social context, style, and so on.

2) Accessibility and Time

When I was writing about casual games (A Casual Revolution), it became clear that two of the primary barriers that prevented people for playing video games were:

  1. Accessibility, broadly understood, including game conventions that players were not aware of, as well as font size, interfaces and so son.
  2. Time. The fact that many games require substantial time commitments, both in absolute time, and in the chunks of they ask you to commit. I argued for interruptibility as an important design principle to reach a broader audience.

These were examples of how game rules strongly include or exclude players, depending on their life circumstances, tastes, and video game experience. Design very concretely matters for who is going to play.

3) The Existential Experience of Playing

I also looked at (The Art of Failure) how failure in a game concretely has implications for our self-image, and how its meaning is tied both to the fictional world, to our identity, and to the social context we are playing in.

This is also where game design observations interface with education. You can design a game, or a test, in ways that encourage or discourage those who fail.

4) When Rules control the World – Gamification & Algorithmic Bias

Finally, I think gamification and algorithmic bias overlap. Gamification sets up measures for what we are supposed to do – such as approve the most loans, pass the most students, publish the most papers, be most active on social media. In all cases, what is measured and rewarded is usually not quite what we actually think of as valuable. Publishing more papers is not actually valuable, but we have just set up incentive structures that reward it, and punish those who work for a long time on one paper.

As the gamification term suggests, I suppose, these are typical game design decisions, where we set up rule systems that reward – or punish – certain behaviors.

This problem overlaps quite closely with algorithmic bias: By now, we all have firsthand experience with how algorithms select which posts to feature in  newsfeeds, and I think most people understand that AI technologies are both invisible to us, often promoted as objective by companies, while in fact of course they embody lots of biases (here is a bit of a reading list) in gender, ethnic, racial, and class[i] in health[ii] [iii], image recognition[iv], loan approvals[v], policing[vi], to give some examples.

This not to say that video game studies invented the study of algorithms, but the study of games helped bring this hitherto technical domain to the humanities, and it is much easier to discuss game rules now that public discourse so often touches on algorithms.

How Video Games Interface with the World

To sum up, we can think about how the (political) meaning of video games can arise in many different ways, and we can think about this works slightly differently for rules and for fiction (narrative):

Rules interface with the world literally: what happens in a game really does happen. If you lose a game, or if the rule structures makes a game inaccessible to a large portion of the population because it is too difficult, requires too much time, or makes too many assumptions about the audience, that really does happen.

Fiction (~narrative) interfaces with the world metaphorically: what happens in a fiction does not actually happen. This does not make it less culturally important; it just means that we (often) see fiction and narratives as questions of representation and values.

[i] Sara Wachter-Boettcher, Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech, 1 edition (W. W. Norton & Company, 2018).

[ii] Ziad Obermeyer et al., “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations,” Science 366, no. 6464 (October 25, 2019): 447–53, https://doi.org/10.1126/science.aax2342.

[iii] Christina Oxholm et al., “Attitudes of Patients and Health Professionals Regarding Screening Algorithms: Qualitative Study,” JMIR Formative Research 5, no. 8 (August 9, 2021): e17971, https://doi.org/10.2196/17971.

[iv] Carsten Schwemmer et al., “Diagnosing Gender Bias in Image Recognition Systems:,” Socius, November 11, 2020, https://doi.org/10.1177/2378023120967171.

[v] Emmanuel Martinez and Lauren Kirchner, “The Secret Bias Hidden in Mortgage-Approval Algorithms – The Markup,” August 25, 2021, https://themarkup.org/denied/2021/08/25/the-secret-bias-hidden-in-mortgage-approval-algorithms.

[vi] Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St. Martin’s Press, 2018).

ToDiGRA special issue on “Teaching Games: Pedagogical Approaches”

For your theoretical traversal: Transactions of the Digital Games Research Association (ToDiGRA) special issue on “Teaching Games: Pedagogical Approaches”

New Paper: The Game of Video Game Objects

I have a new paper out, just presented at the CHI Play’21 conference:

“The Game of Video Game Objects: A Minimal Theory of When We See Pixels as Objects Rather than Pictures.” In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play. CHI PLAY ’21. https://www.jesperjuul.net/text/gameofobjects/ 

We’ve discussed immersion (for and against), but I argue that we’ve overlooked a much more fundamental question: Why and when do we think of pixels on the screen as objects, rather than as pictures of objects?

During the pandemic, I built a game for exploring this question and wrote an accompanying essay. This extends my previous paper Virtual Reality: Fictional all the Way Down (and that’s OK).

The game presents a series of game objects, and asks the player to consider their status:

  1. When would you describe something as an object, rather than a picture?
  2. When do you think of it as the type of object it represents – like a rock or a lamp?
  3. And when would you argue that an onscreen object is the type of object it claims to be, such as a calculator?

The conclusion is not just that games and VR are cultural forms (obviously), but that we judge game objects based on what we are trying to use them for, and game worlds are always designed for particular kinds of uses. And we know this. There thus can be no universal metaverse, only different worlds built for different purposes.

From the abstract:

“While looking to the future, we have overlooked what is right before us. With new technology, haptics, rendering, virtual reality, we have spent much energy discussing immersion and presence, thinking sometimes about current technology, but often about a hypothetical perfect experience or future perfect technology.

In this, we have forgotten something rather fundamental: How do we in the first place decide to see a group of pixels on a screen as an object to which we have access, rather than as a picture of an object? This paper explores this question through a playable essay. At first, we may think that we will identify anything interactive as an object, but the playable essay demonstrates that this is much more complex and pragmatic, and that this identification has three steps – identifying pixels as an object rather than a picture, reasoning about the object as a specific type of object (such as a ball), and identifying it as a real instance of a type of object (such as a calculator).”

Thanks to IVD at The Royal Danish Academy, Nick Montfort, Stefano Gualeni, Pawel Grabarczyk, Dooley Murphy, and Jan-Noël Thon for comments; to Andrés Cabrero Rodríguez-Estecha for visual design; Stephane Bersot for the calculator asset. The project was made with Unity3D and Low Poly Game Kit by JayAnAm.

Eludamos special issue on Playfulness across Media

Here’s the new Eludamos special issue on Playfulness across Media, edited by Jan-Noël Thon.

Game Studies, volume 21, issue 2 is out

For your theoretical journey: Game Studies, volume 21, issue 2

 

Fear and anxiety induced by games are ubiquitous, but unexplored. This article analyzes fear and anxiety inducing mechanics and modalities found in Slender — The Eight Pages. It gives a detailed account of how fear and anxiety are induced; later using it to further explain the way they might be activated by several in-game components.

by Matthew Horrigan

Many essays in game studies deploy the concept of liminality. The term has become diluted with use. However, Turner and van Gennep developed liminality together with related concepts that suggest a richer account of play than liminality captures on its own. This essay discusses the role of the liminoid in connecting players with characters.

 
by Selim Krichane
This article traces the emergence and generalization of the term “camera” in discourses surrounding videogames. The detailed analysis is based on a large corpus of magazines in French and English and on game manuals. This study enables us to renew the traditional narratives on the relationship between cinema and videogames during the 1990s.
 
by Eoghain Meakin, Brian Vaughan, Charlie Cullen
An illustration and discussion of the practical uses of Aristotle’s Poetics when describing video game narratives. What emerges from this lens is the articulation of a cognitive arc for both the player and player character and the mechanisms used to make this possible.
 
by Gregory P. Perreault, Emory Daniel Jr., Samuel M. Tham
This study looks at the case of the mobile, loot box-focused game Final Fantasy Brave Exvius to better understand how and why gamers spend real-world money on in-game purchases. Players are motivated by their community, social identity, and nostalgia for the games of their youth.
 
by Jon Stone
Are ‘poetry games’ a paradox? This article considers the problems inherent in mixing what Astrid Ensslin describes as “two entirely different interactive, productive, aesthetic, phenomenological, social, and discursive phenomena,” charting the differences in greater detail.
 
by Nansong Zhou
Many Chinese players relate to the game character in Travel Frog as if it were their child. By conducting interviews with 20 players from major Chinese cities, this article explores how the relationships between players and this character are deeply rooted in player conceptions of their ideal lifestyle and ideal parent- child relationships.