SYLLABUS SPRING 2016
ARCH 5424: DIRECT CINEMA MEDIA FABRICS *

INSTRUCTOR: Earl Mark ejmark@virginia.edu

TEACHING ASSISTANT: Katie Banach kab5vv@virginia.edu

COURSE DESCRIPTION

Direct Cinema Media Fabrics is an interdisciplinary workshop and seminar first introduced in spring 2013. The cinema has expanded to encompass a greater range of media and methods, transcending architecture, sculpture, theater, animation, scientific discovery and environmental art. We explore methods of capturing moving images, sound and potentially other sensory input. We explore ways to discern key patterns and shapes through coding computer graphics procedures. We then speculatively and creatively transform the input into 2D or 3D virtual or physical forms. A significant part of the class will be devoted to learning to write code in the processing graphics programming language that was developed at the MIT Media lab (see processing.org). The software runs natively on both the Apple and Windows operating systems. The final project may be in the form of video, computer animaiton or a physical interactive medium.

A curated collection of work created with Processing at http://processing.org/exhibition exhibits methods of filtering captured data, video and images, undertaking  its analysis and interpretation, and creatively processing it to create motion pictures and other media fabrications. For example, a video recording, sound recording or a motion capture body suit may be used to collect initial data. The data is processed to discern patterns, forms or movements. In turn, the movements and other observations are creatively translated into some other media such as a 3D virtual model or animation, a movie, a video installation, or a moving fabric wall. 

The course engages three overlapping phases of a data collection to creative process. The phases are 1. Direct Cinema, 2. Computervision and Processing, and 3. Media as Fabric. Some of the specific methods used in each phase are:

Phase 1: Direct Cinema: Workshops explore a documentary moviemaking style conducive to spontaneous discovery and observation. Subjects may include people and their environments, phenomenal studies of light, air, and water changing over time, or other elements of story and place.

Phase 2: Computervision and Processing: Tools are used to accentuate, abstract, analyze and expand upon the captured data and to set the stage for creative interpretation. This phase will be developed hand-in-hand with learning to program in Processing.

Phase 3: Media as Fabric: The processed input data is creatively translated into either physical or virtual media. This may include a movie, an alternative virtual form (e.g., a choreographed animated human figure), of some other type of virtual or physical element that moves (e.g., a building skin).

Each of the three phases run parallel to a series of six exercises that explores technology towards a final cumulative project. The final project and type outcome is developed on an individual basis. It is anticipated that an interdisciplinary group of students will enroll in the class, such as from design, studio art, media studies, digital music, and computer science, and that the kinds of final projects will vary.


REQUIREMENTS

Six exercises for the term lead towards a final project. The first six exercises count for roughly 50% of the grade, the final project (exercises 6 and 7) counts for 40% of the grade. Class participation through attendance in workshops will count towards 10% of the grade. Full attendance is expected.

FORMAT

The technologies of 2D and 3D computer graphics animation, digital video production, motion capture and video processing, and computer graphics programming are the primary technologies used. Digital sound and recording systems will be introduced.  The degree to which this technology is used will vary according to individual student areas of focus. These include character based visual narratives, explicit reconstructions of buildings and landscapes, or more abstract compositions exploring materiality, light, formal or spatial elements, or a responsive physical object.  

To gain a critical perspective, we will discuss feature length movies, independent movies, animation, interactive projects and student work. There will be a few readings in film theory, design theory, and cognitive science. We will also discuss interactive physical installations, their history of use in film and their potential application to architecture and other fields. The text by Reas, Casey and Fry, Ben, titled Getting Started with Processing is the only purchase requirement (see bibliography below), and can be obtained in electronic book form through nook or iBook (recommended) or paper.  The primary software for the course, Processing, is available at no cost from Processing.Org. Secondary use will be made of Maya, also free under and under academic licensing (Maya) from Autodesk. Details will be provided during the first week of class.

In week three we will begin a gradual step-by-step sequence of learning to program with Processing in parallel to working with other digital media. While students with prior experience may be capable of forging ahead, the basic plan for the semester is to work at a relatively modest pace to ensure that there's ample opportunity to learn both the techniques involved and that assumes no prior background.

Note that some of the outside works that we screen or review, as outlined below,  may change depending upon the direction of student work in the course.

SCHEDULE

The class meets on Tuesday and Thursday in room 105 Campbell from 2 to 3:15 p.m. When announced, workshops will occasionally be held in other spaces, including the Robertson Digital Media lab in Clemens Library and in Campbell 107.

* Direct Cinema is a genre of documentary filmmaking. Media Fabrics was the title of a graduate section at the MIT Media Lab.

SCHEDULE

The following schedule is subject to minor modifications to accommodate the pace of the semester and individual student work.

P H A S E 1 _ DIRECT CINEMA
Week 1: January 21 Introduction: Subject Goals and Syllabus & Video, Direct Cinema, Video Camera Control
Lens aperture, focal length, perspective, depth of field, exposure, videotape, video recording technology.
Screening of movie excerpts that portray movement through architectural space over different intervals of time, such as Forty Second Street, In The Street, Daybreak Express, Cinema Verite, Crises, Jazz Dance.
+ Video workshop: Portable digital video equipment, camera care and handling.
> Read Short Excerpt from Arnheim, Film as Art, pp.  8 - 34 (scanned).
> Exercise 1:  in-camera edited video "post-card" of a place.
Week 2: January 26 & 28 Editing: Depiction of Space and Time in Movies

Editing will be discussed in relationship to the treatment of time and space in movies.
Techniques of cutting and pacing, cutaways, jump-cutting, editing, l- cuts, splice-cuts, mise-en-scene.
Movies which exemplify strategies for editing will be screened, including Raising Arizona, Follow The Fleet, Don't Look Back, Primary 
+ Editing workshop: Quicktime editing, compression, digital video capture
> Read Short Excerpt from Arnheim,  pp. 181 - 187, 34 - 87 (scanned as 181- 187, 34 - 57 and 58 - 87)
> Reas and Fry, Getting Started with Processing, Begin reading Chapters 1 and 2: Create first processing program (see online link text below)
Week 4: February 2 & 4 Editing (continued) & Processing: Introduction to Graphics Programming

Build-up of shot->sequence->scene->movie.
Sequences from Employees Leaving Lumiere Factory (Lumiere), Arrival of Express Train at Lyons (Lumiere), Trip to the Moon (Melies), Don't Look Back (Pennebacker), Dead Birds (Gardner), Muybridge.
Optional reading: Hodder, Fraser Harbour, "Cinema Veritas", Harvard Magazine, Vol. 108. No. 2., November - December, 2005.
> Exercise 2: A study of a process
+ Editing workshop: video editing, compression, digital video capture (iMovieHD or FinalCutPro).
+ Graphics programming workshop:  drawing basic shapes in 2D.
Optional Reading Excerpt from Murch, Walter, In the Blink of An Eye, A Perspective on Film Editing, Silman-James Press, 1995, (pp. 1 - 14, 57 - 63).
> Read Reas and Fry, Chapter 3, Define and draw simple shapes.
Week 5: February 9 & 11 Editing (continued) & Processing: Working with Data and Interativity

Digital video formats.
Masking, Chroma and luminance keying.
Transitions such as dissolves, end cuts, v, etc..
Key light, fill light, daylight.
Comparison of animation .vs. video lighting.
Screening of movies where lighting used most critically, such as The Third Man ,The Magnificent Ambersons, Eaux D'Artifice and others.
Optional Reading: Hiltzik, Michael A, "Digital Cinema Take 2", Technology Review, Vol. 5. No. 7., September 2002.
+ Lighting workshop: still life setup as an in-class exercise in lighting.
+ Graphics programming workshop:  processing data and interactive control.
> Read Reas and Fry, Chapter 4, Store, modify and reuse Data
> Read Reas and Fry, Chapter 5 ,Control and Influence programs with mouse and keyboard
P H A S E 2 _ COMPUTERVISION &  PROCESSING
Week 6: February 16 & 18 Processing: Media

+ Graphics programming workshop:  processing media. 
>Read Reas and Fry, Chapter 6, 7,  Load and display media including images

Week 7:  February 23 & 25 Processing: Shapes and Movement
Sequences from Toy Story, Sesame Street, Mary Poppins, Mr. Blandings Builds His Dreamhouse, The Harvey Girls,  
Digital audio recording, microphones, digital effects.
Mixing multiple tracks.
+ Graphics programming workshop:  2D and 3D shape movement and animation
> Optional Read Excerpt from Sacks, Oliver, Chapter 1, The Man Who Mistook His Wife for a Hat, Harper & Row, 1985.
> Read Reas and Fry, Chapter 8, Move and choreograph shapes
Exercise 3: Processing Text and Images
Week 8:  March 1 & 3 Processing: Motion and Video Editing 

 Optional video tutorials - lynda.com iMovie HD 6 Essential Training (slightly outdated)
+ Graphics Programming: Animation
> Read Reas and Fry, Chapter 9 Build new code Modules.
Week 9:  March 8 & 11 S P R I N G   B R E A K
 
Week 10:  March 15 & 17 Introduction to Computervision

Video capture and processing
>Read Excerpt from Lynch, Kevin, Chapter 5, "What Time is This Place", MIT Press, 1972.
>Read Excerpt from Rainer Maria Rilke, "The Notes of Malte Laurids Brigge", 1910, from Existentialism from Dostoevsky to Sarte
+ Graphics programming workshop:  detecting and interpreting motion
+ Arduino Workshops 1 & 2:   leds and light sensors, miscelaneous sensors(UVA Alderman Library, Scholars Lab)
Week 11:  March 22 & 24 Processing: Working with Variables and Functions to Control Graphics Composition

Examples from Processing Web Site
Geometry, Windows and Libraries
+ Graphics programming workshop:  data abstraction and functions.
>Read Reas and Fry, Chapter 9, Combining Variables and Functions.
>Read Reas and Fry, Chapter 10, Working with Lists of Variables
+ Arduino workshop 3: motors (UVA Alderman Library, Scholars Lab)
Exercise 4: Media Fabric Time Machine/Final Project Preview
Week 12:  March 29 & 31 Processing: Iteration and Recursion
Sequences from Toy Story, Jurrasic Park, Titanic, Happy Feet, and Life of Pi (if available)
Conditionals, Iteration and Recursion
+ Graphics programming workshop:  data abstraction and functions.
> Read Reas and Fry, Chapter 11, 3D image export.
Week 13:  April 5 & 7 Processing: Video Motion Capture 

One of the two workshops may be held at Campbell 107
Processing and recording video, comparison with commercial tools (Adobe AfterEffects)
+ Video workshop: video capture and manipulation.
Exercise 5: Final Project Preview
> Read Reas and Fry, Chapter 11, 3D image export.
+ Arduino Workshop 5: reading and interpreting gesture
PHASE 3 _ MEDIA AS FABRIC
Week 14:  April 12 & 14 Graphics Processing: Microcontrollers

Microontrollers and Light, Sound and Temperature Sensors
+ Video workshop: introduction to Arduino and Microcontrollers.
> Read Reas and Fry, Chapter 11, reading and data from an arduino board.

Week 15:  April 19 & 21 Graphics Processing: Microcontrollers

Microcontrollers and Motor sensors
Exercise 6: Final Project
+Arduino Workshop 6: connecting visual imput to motors and other effectors
Week 16:  April 26 & 28 Independent Workshops and Development
Open workshops/ tentaively graphic menus in Processing/microcontrollers and eTextiles
+ Conductive thread and fabric
Week 17: May 3 Independent Workshops and Development
Preview Final project
Week 18: May 12 (tentative, date 2 pm)  Final Project


Selected Bibliography (links underlined below are to UVA VIrgo Electronic Subscription, free on grounds or through VPN access from off grounds)

Animation:
Introducing Autodesk Maya 2015  (NOTE: This link is temporaily missing) by Dariush Derakhsshani, Autodesk Official Press, 2014° §
Mastering Autodesk Maya 2015 (NOTE: This link is temporaily missing) by John Palamar, Wiley Publishing, Inc., 2014 (2016 edition expected August 1, 2015) A° §
Maya Visual Effects: The Innovators Guide by Eric Keller, Autodesk Official Press, 2013 +  §
MEL Scripting for Maya Animations, Second Edition,  by Mark R. Wilkins and Chris Kazmier, Morgan Kaufmann Publishers, 2005 + §
Maya Secrets of the Pros 2nd ed. / John Kundert-Gibbs, Dariush Derakhshani et. al. Sybex Inc., 2005. +  §
Maya 8 for WIndows & Macintosh (Visual QuickStart Guide) by Morgan Robinson, and Nathaniel Stein, Peachpit Press, 2007 §
lynda.com tutorials activated for internal use at UVA are available through this link on netbadge

Graphics Programming:
Reas, Casey and Fry, Ben, Getting Started with Processing, 2015, Media Maker Inc. Ø §
Reas, Casey and Fry, Ben, Processing: A Programming Handbook for Visual Designers and Artists, August 2007, MIT Press. ±
Reas, Casey, McWilliams, Chandler, and Barendse, Jeroen, Form+code in Design, Art and Architecture, August 2010, Princeton Architectural Press. §
Shiffman, Daniel, Learning Processing: A Beginner's Guide to Programming, Images, Animation and Interaction, 2008, Morgan Kaufman.§
Processing Software Download (Max OS and Windows Compatible)  processing.org. Ø

Microcontrollers:
Banzi, Massimo, Getting Started with Arduino, 2008, O'Reilly
Margolis, Michael, Arduino Cookbook, 2011, O'Reilly Media Inc. §
Sparkfun Inventors Guide, Creative Commons, San Francisco, CA (PDF file)

Moviemaking:
Arnheim, Rudolf, Film as Art. Los Angeles: University of California Press, 1957.*
Bazin, Andre, What is Cinema: Vol. I. Los Angeles: University of California Press, 1967.
Bazin, Andre, What is Cinema: Vol. II. Los Angeles: University of California Press, 1972.
Cocteau, Jean, Beauty and the Beast: Diary of a Film. New York: Dover Publications, 1972. **
Greene, Graham, The Third Man and The Fallen Idol, Penguin Books Ltd., 1977.
Murch, Walter, In the Blink of An Eye, A Perspective on Film Editing, Silman-James Press, 1995. **
Nichols, Bill, Movies and Methods. Los Angeles: University of California Press, 1976.
Nizny, Vladimir, Lessons With Eisenstein. New York: Da Capo Press, Inc., 1979. (trans. and ed. by Ivor Montagu and Jay Leyda)
Sarris, Andrew, The American Cinema: Directors and Directions 1929 - 1968. Chicago: University of Chicago Press, 1968.

Philosophy and Psychology of Perception:
Arnheim, Rudolf, Art and Visual Perception. Los Angeles: University of California Press, 1974.
Goodman, Nelson, Languages of Art. Indianapolis, Indiana: Hackett Publishing Company Inc., 1976.
Sacks, Oliver, The Man Who Mistook His Wife for a Hat. New York: Harper & Row Publishers, 1987.
Langer, Susanne K., Philosophy in a New Key. Cambridge, MA: Harvard University Press, 1957.
Putnam, Hillary, Representation and Reality. Cambridge, MA: M.I.T. Press, 1988.
Existentialism from Dostoevsky to Sarte, Edited by Walter Kaufmann, The World Publishing Company, 1956 (Rainer Maria Rilke, "The Notes of Malte Laurids Brigge", 1910)

Sample Vendors for MIcrocontrollers, Instructables, Microcontroller Software, Sensors, Motors and Gears
Arduino.cc -includes free Processing IDE (Integrated Development Environment) for Arduino platform Ø
Adafruit.com
Sparkfun.com
Robotshop.com
scientificsonline.com
stevenshobby.com

Ø - Required (free resources, see Reas and Fry, Processing and Aduino IDE above)
§ - These texts are availble from UVA Virgo for on-line viewing. For a more complete listing of on-line materials see links.
 ± - Highly Recommended Tutorial/Textbook
°  - Recommended Tutorial/Textbook
*  - Highly Recommended Book
**  - Recommended Book