Art & Music - Module 5: Combining Art with Music (Fall 2024)

For Teachers
1

"Creating Art & Music" is an introductory course designed to introduce creative activities in the context of art and music. Thus far the course has been taught at the middle school, high school, community college, and university level (with appropriate adaptations for each level). Each week this fall, students in the course are posting their projects in this strand of the Snap! forum. Here's a link to the course materials:

Art, Animations & Music

This week's module is titled "Combining Art with Music". In it, we will explore the ability of Snap to use sounds in projects. We will then learn how to synchronize these sounds with visual elements on the stage.

In 1940, Disney released Fantasia, an animated film consisting of eight segments set to pieces of classical music performed my the Philadelphia Orchestra under the direction of Leopold Stokowski.

Fantasia-poster-1940
Figure 1. Fantasia theatrical release poster (1940)

The film's style and narrative format proved to be revolutionary in a number of way:

  1. Fantasia was the first major feature film to blend animation with classical music on such a scale. Walt Disney sought to create "visual music" where the animation wasn't just a backdrop but a core element expressing the emotional and narrative power of the music. Pieces like Tchaikovsky's "Nutcracker Suite" and Stravinsky's "Rite of Spring" were used to tell imaginative, visual stories, turning music into visual art.
  2. The film pushed animation boundaries by exploring different visual styles for each musical piece. For instance, "The Sorcerer’s Apprentice" featured Mickey Mouse in a traditional narrative, while the "Toccata and Fugue in D Minor" used abstract, experimental animation, blending surrealism and abstraction.
  3. Fantasia was one of the first films to elevate animated films into the realm of high art. At a time when animation was often viewed as children's entertainment, Fantasia was aimed at adult audiences as well, making the case that animation could express deep artistic and emotional ideas.
  4. Fantasia inspired countless animators and filmmakers to think of animation as a serious art form. It also helped introduce classical music to a wider audience, using visual storytelling to make the music more accessible.

AIFF Fantasia Introduction (3 min) (Includes a brief interview with Walt Disney describing the conception of the film.)

Fantasia: Tocata & Fugue (9 min, animation beings at 3:40)

The assignment for this week is to create an animation with synchronized sound. You may use sprites from the sprite library, import sprites, or create your own in Snap. The animation can be representational or abstract. For sounds and music, we will learn how to record sounds in Snap! and write simple melodies or "motifs" using blocks in the Tunescope library.

Save your program to the cloud, share it so that others will be able to access it, and then post the link as a reply in this strand.

Please provide contextual information for your project. If there is anything in particular that inspired your project, please share that as well.

2

https://snap.berkeley.edu/snap/snap.html#present:Username=tristar10&ProjectName=music%20and%20art
Here is my project of combining art and music: I made a little bit of music for this assignment to see how easy is it to actually write music on snap. As it turns out it is REALLY REALLY hard. I am used to writing in a piano roll and writing on snap is like blindfolding. I have to first use my DAW to write the melodies and chord progressions before I can translate them into snap language by looking at the score. Also the performance is very sad as adding any other instrument is going to crash it by any second. However, I would like to give huge credits to the development team and the faculties. Y'all are amazing to make making music possible on this small platform and it worked pretty well!

3

@tristar10 As you know from our prior discussion, our goal in development of TuneScope was not to recreate a Digital Audio Workstation (DAW) in Snap!. Rather, our goal was to introduce coding to novices in the context of art & music.

If you enjoy developing tracks in your DAW that are imported into Snap!, the easiest way to do this might be to develop a MIDI import procedure. A couple of prior attempts to do this have demonstrated the feasibility. However, importing multiple tracks into Snap! while compensating for differences in approaches to notation presents challenges.

As you note, development of multiple tracks of any length quickly encounters the limitations of this environment. However, as an electrical engineering student, you are in a position to circumvent these limitations by developing a variation of Tunescope that sends its output to a polyphonic music synthesizer. This might avoid the inherent latencies and delays that occur when a web audio font is retrieved via the Internet.

We have a music synthesizer on order that should arrive later this semester if you would like to attempt this.

4

Hi, and thanks for replying. Let me give some more explanation on what is my intention about the comments I made earlier: As our goal of the music section is to create a piece of music by combining all of the elements, I am trying to provide insights for other people what might be the easiest way to make melodies and chords-that is to first write it out somewhere and then do the translation. Also I shared an example of what is closer to the limitation of how full a piece could be before it is crashing. Besides, I've mentioned that people who developed this (i.e. y'all) are amazing in terms of making this happen in snap.
If you take a look at the project including the previous ones that I've made, I've put hours and hours and lots of effort into it just to take it serious at this class. From my understanding of your reply, you are seeing my comments as negative response. However, I am not trying to criticize this by any means. I made a whole paragraph of song that is very complete hence it is hard. If you have any other questions we can talk after Wedensday's class. :slight_smile:

5

@tristar10 No, we weren't viewing your comments as negative in the least - just an acknowledgement of the limitations of the tool. Jo Watts is currently in physical therapy. Once he finishes, he plans to comment in more depth regarding the technical aspects of your composition, which we liked very much.

However, in a more general post, we also wanted to acknowledge the limitations of the tool that you noted, as well discussing possible ways around the constraints of a web-based approach.

6

Here's a link to the data sheet for the music synthesizer chip that we ordered:

Data Sheet (SAM2965)

7

Oh that's perfect. I am a sensitive person and I don't want to offend people. Anyways, I will start looking at the microcontroller after I get my hands on them!

8

Here is the link to my project for this week: Week 6 Project. For the melody part of this week's project I aimed to recreate a very simple version of the music used in the movie Interstellar. To fit with this theme I chose the star costume from the Snap! costume library and tried to have them blink in rhythm with the beat. Additionally, I used the dot procedure from previous weeks to give the piece a feeling of space and vastness as I attempted to connect the visuals to the melody.

9

For my project I decided to recreate the beginning of the song "Another Day of Sun" from the movie La La Land because I love that movie a lot and music is super important in that movie since it's musical-esque so I thought it would be a good subject for this project. I did 4 colored dots: red, green, yellow, blue because in the movie there's a dance/music scene where there are 4 women wearing those color dresses and dancing. There's a shot in the sequence where the camera is filming them from above while they are dancing in the road at night and they kind of (abstractly) remind me of four colorful dots dancing against a black background. I made the dots drift and pulse around simultaneously to kind of simulate them dancing around the stage. One thing I struggled with was timing the speed of the dots to synchronize with the music---I don't know if that's possible to do but if so I'd definitely like to try that!

10

Here is my project. I recreated the song "rain, rain, go away" and used a cloud as my animation by switching between different costumes.

11

For this project, I chose a song from a show I like called Golden Wind. Since the song is fast paced and upbeat, I wanted the visuals to reflect the tempo of the song and the sense of movement. To accomplish this, I create clones which draw across the screen to screen horizontal lines. These lines appear and disappear over the song as they are repeatedly drawn over one another. The lines also draw at different y-values depending on the note being played.

Project Link: Combining Art with Music

12

Here I created my own melody, and I have mapped the notes to the size of my sprite, which is an organic shape. I'm aiming to get a vivid movement using stamp and manipulating the changes of size. I am planning to utilize more colors and make the movements smoother.

Problem: there seems to be a problem with the motion module. Haven't found a way to fix it yet.

https://snap.berkeley.edu/snap/snap.html#present:Username=rachael_yuan&ProjectName=Fantasia%20Design%20Project_Rachael%20Yuan

13

For my project, I decided to recreate the song "Can't Help Falling In Love" by Elvis Presley. For the visuals, I wanted to match the duration of the chords to the motion of the dots disappearing but it was a little challenging trying to get them to sync up. I think once I can get that fixed I'll try animating a sprite that's not just a dot.
Link

14

For my project, I was inspired by Plantasia (very close to Fantasia hehe), an electronic album made specifically for plants. I took the chill, vibey aspects of the music and paired it with my animation of hand-drawn sprites, which included many cats. I transcribed the music using multiple instruments and chords, which was quite a challenge. Snap has its limitations when it comes to handling multiple instruments-like not being able to control volume for individual segments. Also, snap often lags when it comes to playing the music on time on the first try, so I hope to find a way to fix that. I might add more cat sprites for my leftover time as well.

15

For my project I decided to use the precreated motif, Over the Rainbow, and then use animate a rainbow and sun to move with the notes. I was having a lot of trouble getting the lists of notes to work and that made it difficult to access each note individually. I ended up having to use a lot of wait blocks instead (which I know is not the most efficient, but I couldn't figure out my bug). Here is the link to my project: Fantasia

16

I tried to animate my sprite to a version of the Friends theme song intro, however my Snap! Currently will not draw anything due to an error as seen in the screenshot. Not sure whats happening but this is my project! Fantasia

Screenshot 2024-10-02 at 9.51.51 AM

17

I decided to use "A Dream is a Wish Your Heart Makes" from Cinderella as my motif for this project. To accompany it, I created 5 stars that drift, spin, and pulse across the screen according to the song's tempo. I used an image of a night sky from Disney's old animated intros to their movies to evoke the feeling of Cinderella dancing under the stars at the ball in the original movie.

Here is the link to my project: Combining Art with Music

Note: To make the stars sync up properly, the green flag button may need to be pressed again once TuneScope is finished initializing.

18

I've decided to make a drum beat as my main motif trying to mimic some famous motifs from the past. I haven't incorporated any visuals yet as I'm still trying to get the music set in stone. Heres the link: Fantasia - Spencer Lewis

19

@reecemullahy

I think the effect you've created here works quite well. As mentioned in class, it's impressive that you managed to work the timing out so well, but there are a number of ways we could link the blinking of the stars directly to the notes being played.

One way would be to use the Current Note block, create a variable called New Note, update new note to be the same as current note until the next note plays, and then monitor the variable to detect when the two are equal and when they aren't.

However, an easier way would be to place a toggle switch directly into your code.

Snag_fc71678

This block of code uses a variable named Switch and simply flips it back and forth between true and false whenever the block is activated.

If you place this block after Play Note and Wait in your Play Motif block, it will toggle the switch after each note.

Then, if you update Blinking Star to include an if statement inside of a forever loop, you could constantly check to see whether the switch is on or off. When it is on, tell the stars to show. When it is off, tell them to hide. This is probably the simplest solution.

20

@francesdai

Love the overall sound and effect, but you would likely have an easier time tying the animation directly to the music if you used listed of notes instead of coding out lines of Play Note and Wait blocks. To see how to do this, take a look at the Sample Motifs linked on the website for this week's module under Sample Code. By inputting a list of notes and note durations to the for each loop, you can use the item of reporter block to select notes from the first column or note durations from the second column.

The Current Note block will automatically report the note currently being played, so you could use a When starter block to initiate a certain action in your animation sprites when Current Note equals a specific note (e.g., C4).

You could also create a similar block for note durations by setting a variable to be equal to the note duration just played every time the for each block iterates to the next pair of notes and note durations. For example, a your animation sprite might constantly check variable named Note Duration to see if it equals a quarter note and then perform a specific action when that check reports true.

Unfortunately, the Play Note blocks do not directly incorporate a rest, so you will still have to break the music into chunks on either side of the rest blocks. We'll address this issue in a later module and show you how to solve the problem using tracks.