EDP Journal – Stephen Heymann

Final – EDP Logo Generative Animation

I’m pretty happy with how my final project turned out. We were asked to bring in a few generative animation examples to use as creative inspiration for our final project. I brought in this example, and it didn’t take long for us to crack the code of how it was accomplished in class. I took this resampled code and started working on expanding it’s premise which was pretty basic. Understanding classes and how to combine them with arrays was pivotal for developing this project.

Chris showed us how to detect image attachments in p5js by telling a particle to sense the image by color percentages. This opened up a new possibility to animate a ton of pixels and mask them until they sensed certain images that I preloaded in the sketch. Here are the 4 images I designed, each had a different class of particles interacting with them to create a multilayered effect:

By using variable speeds that were linked to perlin noise I was able to create some pretty fun effects. Perlin noise is very sensitive, so dialing in the right numbers took a very long time. There are 4000 total pixels at work in this animation so it gets a little bogged down, as they are constantly searching for the red to trace in the image examples I posted above. I’m happy with my progress in p5 for this quarter and am excited to keep digging into programming.

Arduino & Potentiometers

EDP Journal

We started messing around with Arduino, a type of microprocessing kit that can be programmed to control devices. I’m pretty illiterate when it comes to electricity and how connections work, so this is a good opportunity for me to gain some basic knowledge of circuitry. We plugged a few things into our breadboards, made lights flash and then starting messing with a sliding potentiometer. This is where my interest is peaking, because I may be able to program a couple of knobs to interact with my final Sonic Arts animated projection. I’m pretty nervous about hooking this stuff up to my laptop, as we were warned a bad connection can fry your USB port.

I was provided a tutorial on how to connect p5js to a serial input, which is what I will need to do if I want to make my animation interactive. I really need to run through some more experiments with my SparkFun Kit to get comfortable with this stuff and still feel a little lost. We were provided a really nice introduction tutorial for the kit here.

Meanwhile, I have done a rough p5js sketch of my Sonic Arts projection, the only thing I’ve animated is some twinkling stars. The idea would be the blue ring would orbit around the planet and pulse to the music’s amplification levels. Ideally, users would be able to turn knobs and effect the color and intensity of the animation.

p5js – Perlin Noise, Audio & Functions

Creative Coding, EDP Journal

Screenshot from my Perlin Noise driven animation, which syncs with any accompanying soundtrack's amplitude.

Click the image to view the animation. // Screenshot from my Perlin Noise driven animation, which syncs with any accompanying soundtrack’s amplitude.

We messed around with Perlin Noise in p5js last week and I got pretty excited at the possibilities of random generative animation. I kept messing with this concept and have a couple animations set up to link with audio. The amplification of the audio basically affects the speed and size of the animated shapes. I’m pretty shocked that I’ve gotten to my goal of tying these elements together, and will hopefully have p5 incorporated as an animated, interactive projection into my final audio piece for Sonic Arts.

Initial Robot Head Sketches

Moving on, we jumped into creating custom functions which I can already tell is a powerful tool. Our project was to create a 20×20 grid of 4 different robot heads, with 4 different audio cues tied to clicking with the mouse. I’m happy to say I wrote the code without a lot of hangups, but I got stuck on assigning the audio cues.

Click the image to load the robot audio sketch. // 20 x 20 Grid of talking robot heads.

For “mousePressed” I nested my “if” statements and rendered them inoperable, but got it fixed up with my instructor’s guidance *wink wink* and some help from my colleague, Rich. I’m trying to go the extra mile and make every individual robot head clickable, as of right now just each column is clickable.

Click the image to view the animation! // My first time writing a custom function, I like these glowing Halloween Heads.

I’m looking forward to learning Arduino this week, as it may be my gateway into figuring out a way to give my Sonic Arts installation that extra level of interactivity through touch activation. I want to tie my p5js code to physical objects that will signal the animation to change color and amplitude. We will see if I learn enough to get it done…

p5js – 2

EDP Journal

We have moved on to combining for loops and arrays in p5js. This is a quick way to multiply a ton of objects and get them animated. I’m kind of understanding the basic idea, but still cannot seem to build out a sketch without referencing a tutorial or help website. Every animation or function has a glitch and executing what I think are simple functions is taking forever right now. Hopefully I get more fluent with practice. My main issue is that I have conflicting for loops, one will not allow the other to run, and I have no clue how to fix this. I’m also still trying to clarify how to write conditional statements correctly the first time around.

What you see above is a limited version of what I was trying to accomplish, still kind of fun to look at though. The ellipses are bouncing back and forth as they touch the edges of the canvas, changing color depending on which side they bounce off of. I have a couple growing and shrinking rings that mask the snow globe effect to create a little more visual interest. The rings are supposed to be fading in and out, but I can’t get the “if” statement to work as of yet.

I’m hoping to get this syntax down quickly, because I’m thinking of incorporating a looping p5js animation into my final Sonic Arts sound installation. It would be really nice to have a projected dynamic animation accompanying my sound project, so I’m diving into sound interactivity tutorials with Dan Shiffman. Am I going way in over my head? Probably. Right now I’m looking at amplitude analysis so I can make my animations visually react to a soundtrack.

Mario Pong

EDP Journal

Click in the window to play. A and Z keys move the left bar up and down.

Programming in p5.js

EDP Journal

We have moved on from video production to programming in p5.js, and I’m super excited about diving into this subject. After a couple of assignments on just sketching some objects in p5, we have moved on to working with interactive features. We built a basic game of pong in class and the goal now is to modify the game to have it’s own custom theme. I chose Mario Bros as my backdrop, turned the paddles into some bricks and substituted the pong ball for a turtle shell.

To make the paddles (bricks) bounce the ball, parameters are set on either side of the paddle. While this makes sense, I cant seem to figure out how to set parameters around the other objects in the field of play (the ? boxes, brick, and pipe in the lower right). I want the turtle shell to bounce off these objects, but no luck setting parameters around these objects as of yet…

I also want to include the classic Mario Bros. soundtrack, so I’ve been doing a string of sound tutorials from Daniel Shiffman to cue some music as well as sound effects when the shell hits the paddles (going to use the CHA-CHING noise when Mario hits the ? box with his head). So far I have the soundtrack working with a play/pause button, but still working on the cued sound effects.

My biggest hangup with programming is the compounding math that is involved in functions. It’s not difficult math, it’s just that numbers scramble my brain and slow me WAY down when critical thinking is at play. However, I’m already noticing that the more I tinker in p5, the more intuitive it becomes and the quicker I can pick up on concepts.

Speculative Fiction – Mind’s Eye

EDP Journal

Here’s the final product. Enjoy!

Final Round – Speculative Fiction

EDP Journal

After getting some much needed feedback from my class, the video seems to be on the right track but still needs a few adjustments. It’s not until you show a video to people that you realize how LONG scenes can take to resolve, so I am definitely trimming the fat where needed.

A lot of the audio pairing came out well, I have the Operating System making obnoxious error sounds when it starts detecting harmful actions that the user might be taking. Our guest instructor suggested I ramp up the intensity of the conflict between the user and the OS and also establish a deeper familiarity between the two. The closing scene where the OS takes control of the user’s legs needs the same treatment of intensification. I am running short on time for edits, so hopefully most of these enhancements can be made through audio. I wanted to share an AI chatbot video I saw a couple years ago that illustrates how quickly AI can escalate a casual conversation into a full blown metaphysical debate at rapid fire speed. Their logic enables them to draw conclusions immediately, so it’s not far fetched for my video to portray the same level of escalation. I also am incorporating a musical piece I developed in my Sonic Arts class to help build tension in the final scene.

I have some major lighting continuity issues in the last couple of scenes, so I’m using this link for color correction in Adobe Premiere to hammer out some color corrections and balance out the lighting and color differences. Note to self: Don’t shoot the same scene at different times of day, the shadows make a HUGE DIFFERENCE. (งಠ⏠ಠ)ง

I have to admit I’m not entirely happy with the film… I feel like I went overboard with a scripted storyline and find it somewhat predictable (I have watched it about 1000 times now to be fair). I would like to try and take more abstract approaches to future projects and leave more open for interpretation and reflection. My paintings tend to meld the abstract and literal, maybe thats an angle I can take next time.

Speculative Fiction – Round 3

EDP Journal

I’ve been reviewing footage for my short Speculative Fiction film and I am seeing opportunities to add some visual interest to the storyline. I’ve gone back and shot all of my scenes a second time, but from different camera angles. This should help break up the monotony of each scene and keep the story flowing at a quicker pace. I’m also visualizing a couple of different quick-cut techniques to show the Operating System (antagonist) slowly becoming fearful, angry and self-aware. Right now I’m experimenting with altering small 1-2 second clips throughout the video switching to the O/S’s viewpoint through the User’s eyes. This video is a good illustration of the feel I’m going for, minus the ridiculous amount of CGI.

(UPDATE – 10/1) After showing my rough cut edits, the class unanimously agreed my 1-2 second frame alterations were a little overkill, and most of the tension building in the storyline could be achieved with audio effects. I agree with these comments, and am starting to realize keeping it simple is the best approach with an amateur film project.

Miriam Petruzzelli’s review of You, Robot (by Riccardo Notte) opens with a similar question that I am trying to address with this project: “What is natural and what is artificial? What characteristic does this distinction take on today, if there be one possible?” I want highlight the ambiguity of what artificiality means when we start installing smart or symbiotic technology into our bodies. We may be headed towards a major shift in self-identification.

Audio an aspect I’m still tinkering with. I have gotten a couple cool effects to represent the O/S’s voice.. I wanted the O/S voice to be my voice, but with a slightly adjusted timbre. I feel like this will present a more intimate link between the User and the O/S. I am working on recording my own voice as well and just rented a field recorder to hopefully get some cleaner audio. I am not happy with the outdoor audio’s ambience either, so I’m scrapping it and replacing it with a much more controllable outdoor track I nabbed off of freesound.org. I’m hoping the recorded dialogue will help bring this film to life.

Speculative Fiction – Screenshots

EDP Journal

A few screenshots of film editing in progress. I removed the outside noise from the recordings and am replacing it with more ambient sound. The audio will be an important part of this film, as most of the piece is dialogue between the user and his Operating System.

(<– Click the image to scroll through or see larger pics)

ლ(ʘ෴ʘლ)

Reading Response #1

EDP Journal

EDPX 4010

Reading Response #1

Reading #1 – Introduction to “Program or be Programmed” (Links to an external site.)

What is your initial response to the argument that we should all know how to code?

Although I think this is a tall order, especially given that a majority of humanity has no major care for the understanding of coding and programming, I do believe the assertion is correct. Not understanding the language of a technology that is beginning to dictate our social behaviors could spell disaster for the human race. The historic examples of human participation on the output side of technology is compelling (we don’t write, we read; we don’t make TV, we watch it).

What is in the space between these polar notions (program OR be programmed)?

The power of personal judgement and awareness. A human’s ability to disengage from an imbalanced and poorly conceived system or program. Just because a person cannot communicate to machines through programming doesn’t mean their reaction (or nonreaction) to the program presented is not vital.

How should we alter our education system to account for Rushkoff’s (or your) proposed future?

Introductions to technology should begin with physical manufacturing and software manipulation. Kids and adults alike today simply accept the glowing screens and apps in front of them with no understanding of how they came to be. If students had to build their own PCs and phones from basic components and implement open source programs to facilitate their education, these tools would be understood as such, and not magic machines that dictate our lives.

Reading #2 – To_Save_Everything_intro (Links to an external site.)

What is your initial response to the “Brave New World” that the author discusses?

Eerily frightening, as if the human element has been removed from everyday activity. We have free will to change our habits, the way we think and act. As these smart solutions become implemented, I get the sense that people will be more apt to stay locked into their own highly customized realities and become dependent on large tech corporations to identify moral imperatives.

What are some recent technological advances that suggest this is the future we are heading towards?

Any and all apps measuring your daily “achievements”. Fit-Bits, smart phone apps to sense your quality of sleep and chips to measure your driving habits that communicate with insurance companies. GPS tracking has become prevalent in most new apps, prompting users to constantly inform whomever about their habits and whereabouts.

Who is embracing this future and why? Who is not and why?

Who is: My guess is anyone working and profiting from the tech industries that are shifting to a more philanthropic focus. Profitability can alter people’s belief systems rapidly. The working class Baby Boomer generation who has become accustomed to technology that increasingly makes life more convenient and practical. Users of technology with no programming or networking knowledge. Newer generations being born into the internet-age and the normalcy of sharing each other’s personal information.

Who is Not: Programmers because of their basic understanding of operational tendencies. Non-users and disengaged fragments of modern society more interested in holistic practices.

Speculative Fiction – Script Ideas

EDP Journal

First project: Write a script and storyboard a speculative fiction that challenges current technical pathways. This will eventually become a short film. I have the option to present a couple of ideas, so why not take advantage? One idea is relevant for current emerging tech, the other is a little far-fetched. Here they are:

Script Concept 1:

Augmented Reality: Contact Lenses and Visionary Implants
- I have read about “smart lenses” and of course the emerging bionic eye surgeries that will no doubt become more refined and safe in the near future. This is my take on what could happen if smart lenses and chip implants become the convenient new way to interact with visual media.
- Possible pitfalls to depict:
  - Viruses (porn sites pop up in your visual field when giving an important presentation)
  - Hackers watching your visual activity live (and broadcasting it)
  - Obnoxious advertising because you didn’t pay full price for your app.
  - Increased accidents and deaths, people walking into each other and traffic.
  - Bleeding eyes and seizures. A new line of eye drops and designer drugs are developed to mitigate these effects.
  - Confusion between augmented vision and reality.
  - Loss of desire/drive for human contact and interaction.
  - The world becomes a playground or setting for FPS games. Warehouses are transformed for safe environments to play. Scores for popular games give rise to star status. Point accumulation is associated with attractiveness. HUD in
  - Part of the culture is lazy, fat & complacent, part of the culture is incredibly fit, active and domineering.

Script Concept 2:

Personal Portals
- A world 100 years in the future where portals are accessible to nearly every modernized human. Human matter is still limited to traveling a finite distance before being compromised (250-500 miles). Install a piece of portal tape in any doorway in your house to create a portable portal gateway. Credits must be used per transfer, limited baggage. Portal gateways exist all over the world, sometimes in remote locations to facilitate long distance travel to exotic destinations. Space station portals get people to the moon and Mars.
  Issues: Human matter degredation; two people merging through a portal and becoming conjoined; portal suicides; portal malfunctions (misplacement, breakdowns during transit, power loss, inaccessibility to lower income classes). People become bored of travel, no place holds special meaning. Degradation of cultural history. Homogenized language, currency and product offerings.