EigenTV
Summary
As pieces of the Semantic Web continue to trickle into everyday usage, procedurally generated art will reap the rewards of the huge machine-understandable knowledge and content base that the Web is becoming. The goal of EigenTV (name inspired by EigenRadio) is to create the first artistic/entertainment-minded Web agents using Semantic Web-inspired methods. EigenTV is a 24/7 stream of procedurally aggregated audio, video, and text, broadcast in Windows Media format. The video will be made up of found video and images, and the audio will be a combination of streaming audio, found sound bytes, and text-to-speech. The result will a audio and visual collage, organized around a theme (discussed below), generated in real-time, as the user watches. Unlike EigenRadio, EigenTV will have more than one algorithm to produce content. The broadcast will be split into time slots, like (or, as a parody of) traditional television networks. Initially, there will be three time slots. Each represents one possible strategy for creating EigenTV algorithms, or "shows".
- Morning - (Midnight to 10PM) "Call and Response " - This module uses RandomProxy to message a random AIM user, and then uses their response (or any other IMs it receives) to find images and music that relate to the messages.
- Daytime - (10PM to 6PM) "OprahTron" - The daytime gossip talk show, plucked from the pages of blogs, bulletin boards, and social networking sites, hosted by OprahTron: the robot who aggregates the content.
- Evening - (6PM to Midnight) "Prime Time" - A parody of prime time shows. Each hour, a genre and subject are selected. The genres include sitcom, drama, or news/documentary. The subjects will be provided by viewers on the previous day.
At the EigenTV website, users will be able to rate segments of content based on criteria such as "cohesiveness", "visual quality", and "interestingness". Over time, EigenTV will learn what sources, keywords, and collage techniques create the most entertaining content.
EigenTV will be designed in such a way that new modules can easily be added at any time, and the lineup can continue to grow. It will use the Switchboard Toolkit to assemble the content. Switchboard is a Java toolkit designed for real-time artists to facilitate the use of network and web services, including SOAP, REST, screen-scraping, etc. I created Switchboard as my master's thesis at the Information Design and Technology program at Georgia Tech, and this grant will allow me to continue to work with it, and improve it for other digital artists who want to use Internet sources in their work.
Misha Peschan will assist me with some of the visual design and Graham Reznick will help with the streaming video technology.
Prototype
A series of screenshots from the EigenTV demo in which you can see six videos moving across the screen. At the bottom is the keyword that was used to find the content. It seems the words "attack" and "wearing" were taken from the sentences: "The attack occurred at the Buratha mosque..." and "Two of the bombers were wearing suicide vests." Click for a larger version.
EigenTV Demo (10.3M Windows and Mac executables)
This zip file contains a small application (build with Processing - includes source code) that uses Switchboard to parse cnn.com and pick out the important nouns in the first section of the site. These nouns are then used to search for videos with the Yahoo Video search, and for books and DVDs on Amazon.com. The videos are then played in succession as they wander around the screen, and the images from Amazon are tiled in the background. This is all done in real-time, so you will get an entirely new experience every time the program is launched. Due to bandwidth and processor limitations, I could not have more than one video on the screen at once, or include the MP3 stream search feature. These limitations would be eliminated with a powerful server and a fat internet pipe, as I would have on a dedicated server machine. This is a very simple demo with an extremely simple selection process, but it shows the basic technological components involved.
Timeline
| Research and Design | Deliverable | |
|---|---|---|
| September-October | Look into Windows Media Streaming Server and create a development strategy in the form of a framework for the three shows. Determine hardware requirements based on the demands of the framework. | A simple test-broadcast and a hardware setup. |
| November - December | Formalize design aesthetic and flesh out the three algorithms. Create pseudo code outlines. | Design document outlining the three shows and exactly how they will be generated. |
| Development | ||
| January | "Call and Response" algorithm | One functioning module. |
| February | "OprahTron" algorithm | One functioning module. |
| March | "Prime Time" algorithm | One functioning module. |
| April | Streaming Media integration | All modules broadcasting in WindowsMedia format. |
| May - August | Web voting interface integration | Website with functioning voting system. |
Budget
This is a rough initial budget and can be adjusted to suit a different development model.
| Dedicated Windows Server (1 year) | $3,300 |
| Design consultant | $3,000 |
| Video consultant | $3,000 |
| Artist fee | $20,000 |
| Domain name | $40 |
| Total | $29,340 |
The Artist
Jeff Crouse is a digital artist and programmer with an MS in Information Design and Technology from Georgia Tech. He creates and writes about digital art that uses live data sources, and has created a toolkit called Switchboard to facilitate this type of art. He has worked for over 6 years in the web industry, and is an experienced Java programmer.
CV: http://jeffcrouse.info/cv.html
Portfolio: http://www.jeffcrouse.info
Relevant Work
- Interactive Frank (Java application, completed April, 2006)
Interactive Frank is an audio montage generator that constructs a story using a constrained writing technique called "n-gram" and text found all over the web. The story is read aloud using an online text-to-speech service, and the program analyzes the text to find an appropriate live audio stream which plays in the background. It was built using the Switchboard library.
- WebFight (Java applet, completed December, 2005)
WebFight visualizes the amount of text and external links on a given page. Each link on a page spawns another node on the perimeter of its parent, traveling four levels deep per site. Blogs are particularly interesting in this visualization.
- Switchboard: A Real-time Art Toolkit (Processing library, still in production but available for download)
For my masters thesis at Georgia Tech, I studied works that are generated in real-time using live data sources (real-time art), and the values and strategies that these works share. Guided by this research, I created Switchboard: a toolkit for art that is driven by external data sources. It is written in Java and designed for the Processing IDE, an open source programming environment for artists and designers.
Thesis Design Document : http://www.realtimeart.com/thesis
The Processing IDE: http://www.processing.org (not my work: provided for reference) - Advanced Audio Listeners League (started Summer 2002)
My Shoutcast radio station. Listed here as an example of my experience writing scripted scheduling systems.
-
Radical Software Group intern (Summer, 2003)
As the first official RSG intern, I worked on the OS X version of the Carnivore client. In the process, I learned quite a bit about network programming. I also created the first Flash Carnivore template.
Extended Description and Future Plans
EigenTV is the first step towards a procedural media player in which the code that makes up the "modules" would be sent to the user as video is now. The procedural media player would then construct the content in real time, using the viewer's computer. These algorithms could be tweaked at runtime to produce a personalized experience. Currently, the average users' home PC processor is not powerful enough to manipulate multiple audio and video streams, and their internet connection is not fast enough to pull them all down. Eventually this will all be feasible, but until then, we will have EigenTV.
By using new media techniques in a format that is traditionally associated with television, EigenTV is a form of remediation, as described by Jay Bolter and Richard Grusin. Other inspirations include [PAM] and all of the works listed here.
