When it comes to general-purpose microcontrollers, no one expects them to perform heavy software operations. Microcontrollers are simple devices that control binary devices and communicate through serial interfaces. Never try to build audio/video or multimedia applications on microcontrollers. It is challenging and impractical in most situations to handle any type of multimedia through microcontrollers. Microcontrollers weren't just designed for this.
Despite the incompetence for applications such as audio and/or video, it is worth trying and experimenting with audio and graphics projects with microcontrollers. We have already seen how Arduino can generate digital audio in the form of musical notes and melodies. Can Arduino also play music for us? Well, Arduino is unable to play the audio files by itself. It is possible to play audio files in MP3 and WAVE format by Arduino using only external breakout circuits.
The Arduino interacts very well with any speaker, directly or through an amplifier circuit. It could very well make any speaker play music on its own. So how complex can we play on Arduino. We cannot load audio files into Arduino. Your memory and file system don't support this. We cannot use binary code audio for Arduino firmware because flash memory is limited.
So there's only one way to play complex musical melodies on Arduino: streaming. In this project, we will play some complex, multi-track songs on the Arduino, transmitting the contents of the audio file to the board. We will stream musical melodies for multi-track songs using a Python script on a PC/Mac and let the Arduino play them on a speaker.
The project
In this project, we will build a Python application to stream the contents of an audio file from a computer to an Arduino. The Arduino board is programmed to receive the audio stream serially and play it back to the speaker in real time. This way, we will play theme songs from popular films such as Mission: Impossible, Pirates of the Caribbean and Titanic on the Arduino. Following this project, melodies of any song can be transmitted to the Arduino.
Note that the maximum baud rate for serial communication on Arduino is 115200. Most audio files have a higher bitrate, such as 192K. This is why the music streamed on the Arduino may not be the same as that experienced on computers, phones, and other multimedia devices.
There are two versions of this application. In this project, we will build a version suitable for playing single-track audio files.
Required components
- Arduino UNO/Mega x1
- Speaker x1
- Connecting wires/jumper wires
- A computer
MIDO Python Library
We are using Python's MIDO library to extract content from audio files. The library will extract notes and note durations from the MIDI audio file format and transmit the values to the Arduino. The Arduino will then fetch pairs of values representing notes and note durations through a continuous serial stream of data and play the respective melody in real time on a speaker.
Installing the MIDO library
To install the MIDO library on your PC or Mac, run Terminal on Mac or Command Prompt on Windows and run the following command.
py -m pip install mido
The command will install the mido library. It will also install numpy as a dependency if it is not already installed on the system.
Inspecting audio file by Python script
Let's start by writing the desktop application first. The application is written in Python and uses the MIDO and PySerial libraries. The MIDO library decodes audio files and the PySerial library transmits data from audio files to Arduino via serial communication. You will need an intermediate file of the song or song you want to play. You can download an intermediate file for your favorite song, theme or melody online, or convert your favorite song/melody from MP3 format to MID format using an online tool. Before writing the main application, we need to inspect the audio file through Python scripts.
We will play the theme song “He's Pirate” on the Arduino in this project. The MP3 file of this theme song is attached below.
The MID file of the downloaded theme song is attached below.
(Link to poctheme.mid)
poctema
Place the two files in a project folder and create a file with the name “mido-single-track-inspection.py” in the same folder. We will write a Python script to inspect the audio file in this archive. So the main application will be programmed.
Add the following code to the “mido-single-track-inspection.py” file.
import mido
mid = mido.MidiFile('poctheme.mid', clip=True)
mid.tracks
print(len(mid.tracks))
Save and run the script. It will return the track number of the 'poctheme.mid' audio file. There are three tracks in our audio file. If you are working with a different audio file, substitute the file name in the code above.
Let's now inspect the content size of each track. Run the following code in Python script to get the content size of all three tracks.
import mido
mid = mido.MidiFile('poctheme.mid', clip=True)
mid.tracks
print(len(mid.tracks(0)))
print(len(mid.tracks(1)))
print(len(mid.tracks(2)))
We can see nine lines in track 0, 1,285 lines in track 1, and 1,557 lines in track 2. Let's inspect the contents of each track. Run the following code in the Python script to view the contents of track 0.
message=
for m in the middle.tracks(0):
message.append(str(m))
to m in message:
print (m)
As we can see in the screen dump above, track 0 only has the metadata related to the audio file.
Run the following code in the Python script to inspect the contents of track1.
message=
for m in the middle.tracks(1):
message.append(str(m))
to m in message:
print(m)
As you can see in the screenshot above, track 1 contains the audio data. After the 10th line of the track, audio data is present such as note status, channel, note, velocity and note duration.
Run the following code in the Python script to inspect the contents of track2.
message=
for m in mid.tracks(2):
message.append(str(m))
to m in message:
print (m)
As you can see on the screen, track 2 also contains audio data, but for a different channel. There are two channels as this is stereo audio. Audio data from both channels are stored in the audio file on different tracks. We need to extract the audio data from these tracks to transmit to the Arduino.
Note that an audio file can have multiple tracks, each containing audio data for multiple channels. Tracks or channels can correspond to each instrument played in the song.
Let's work on track 1 of the audio file. We need to extract the audio data for streaming. So first, let's drop the meta information stored in the first ten lines of track 1. Note that we have already dropped each message from track 1 into a Python 'msg' list. Add the following code to get rid of the track meta information.
for m in range (10):
from message(0)
Now we need to dump each individual message/track line element into an array. Add the following code to dump each attribute of the track's messages into an array.
grade matrix =
to m in message:
line = m.split(” “)
notesmatrix.append(line)
With each attribute of track 1's messages dumped into a two-dimensional array, we can now extract all the notes from the audio track into another array using the following code.
notes =
for row in the notes matrix:
notes.append(line(2).split(“=”)(1))
print (notes)
The note durations of all notes in the track can be dumped into another array using the following code.
notations =
for row in grade matrix:
notations.append(line(4).split(“=”)(1))
print(notateddurations)
We need to extract notes and note durations from the audio track to play a melody to the audio file on Arduino. Since the audio data will be transmitted serially and played back in real time, each note must be followed by the duration of the note in the serial data. Therefore, we will extract pairs of notes and respective note durations into a different array, so that Arduino will immediately play each note upon receiving pairs of notes and note duration values in real time. The following code stores each note followed by the respective note duration in an array.
musical notes =
for row in grade matrix:
musical notes.append(line(2).split(“=”)(1))
musical notes.append(line(4).split(“=”)(1))
print (musical notes)
Finally, we have the audio data for track 1 dumped into an array. We just need to transmit the contents of the array serially to the Arduino to play a song.
The Python script “audio-single-track-inspection.py” to inspect the audio file “poctheme.mid” finally has the following code.
The desktop app
After inspecting the audio file, we can now write the main application code that extracts audio data from track 1 of the file and transmits it serially to the Arduino. Before that, connect the Arduino to your computer and check the port number it is connected to. In our case, the Arduino connects to COM6. Create a new Python script file and name it 'python-audio-streaming.py'. Add the following code to the script file.
The above script is our main desktop application for streaming audio tracks to Arduino. It uses the MIDO library to extract audio data from a track of the audio file and the PySerial library to transmit the data serially to the Arduino. If you don't have the Pyserial library installed, install the library by running the following command in Terminal/Command Prompt.
py -m pip install pyserial
Circuit Connections
We need to prepare our Arduino to receive the audio stream and play it on a speaker. Connect the Arduino to the computer and connect a speaker directly to the Arduino. The speaker used in this project is a 3-inch 4 ohm, 50 Watt speaker that does not require any amplifier circuitry to generate audible sound. The speaker has two terminals. Connect one terminal to the Arduino ground pin and the other terminal to any GPIO. Here the other terminal is connected to the D8 pin of Arduino UNO.
Arduino Sketch
After making the circuit connections, upload the following sketch to the Arduino.
The sketch reads a couple of bytes of serial data such as a note and note duration and outputs the note to the speaker immediately. This way, while it receives audio streams, the stream is played to a speaker by the Arduino in real time.
How it works
The 'python-audio-streaming' Python script on the desktop takes an audio file and extracts the audio data from one of its tracks. Audio data is transmitted via serial communication to the Arduino as note pair values and note durations. The Arduino receives information about notes and note durations through a serial data stream from the computer. It uses stream data to play musical notes over a speaker in real time.
How to test the project
After writing the desktop application – 'python-audio-streaming', making circuit connections for Arduino and uploading a sketch to the Arduino board, you have the Arduino connected to the computer. On your computer, run the Python script 'python-audio-streaming' and immediately the Arduino will start playing the song melody on the speaker. The transmitted audio data can be viewed in the Python console.
Results
Conclusion
We can play melodies for any song on Arduino by transmitting the audio data to it. Uploading an audio file to the Arduino file system is impossible. It is also impossible to write audio data to your firmware due to the microcontroller's limited memory. In such cases, long and complex musical melodies can be played on the Arduino by transmitting the audio data to it. The audio stream is then played back in real time by the Arduino. The music played will have a lower bitrate than the original audio file, as the UNO's maximum transmission rate is limited to 115,200 bps. In this project, we built a Python script to decipher audio files and stream the audio content of one of their tracks to the Arduino. The Arduino can only play one tone at a time. Therefore, the melody played by the Arduino may not be exactly the same as that experienced when playing the original audio file. Still, if the audio file has a single track or a prominent track, the melody played by the Arduino will roughly match the original MP3 audio.