Text and boxes will only take you so far. This chapter shows how to display pictures, and how to play sound and video. All of them use the raw data you met in the chapter on values: your program reads a file, and sends its bytes straight to the console.
Here's the helper function from the chapter on values again, which sends a file as raw data. You'll use it in every example in this chapter:
import sys
def print_raw(filename):
with open(filename, 'rb') as f:
data = f.read()
print('[' + str(len(data)) + '{', end='', flush=True)
sys.stdout.buffer.write(data)
sys.stdout.buffer.flush()
print('}]', end='', flush=True)
The flushes make sure the text and the raw bytes reach the console in the right order, since Python
sends them by slightly different routes.
An img tag displays a picture. Its primary field holds the picture file's data. PNG, JPEG, GIF (including animated GIFs) and SVG pictures all work.
print('<img ', end='')
print_raw('photo.png')
print('>')
A picture appears at its own size unless you give it a width and height, in which case it's stretched or shrunk to fit:
print('<img width=240 height=160 ', end='')
print_raw('photo.png')
print('>')
Give an img a name, and you can replace its picture whenever you like, just like replacing the text in a span:
print('<picture ', end='')
print_raw('another.png')
print('>')
A big picture takes a moment to send. While its data is arriving, the console draws as much of the picture as it has so far, like a web browser loading a photo. When the picture is replacing one that was already complete, though, the old picture stays on screen until the new one is ready, so the user never sees a half-drawn picture where a whole one used to be.
Sometimes a file arrives in pieces, for example while it's still being downloaded. You can push each piece into the img as it arrives, using .push, and the picture is drawn bit by bit as the pieces come in. But the console has no way to know which piece is the last one. So when you've pushed it, tell the console with .commit:
<picture.clear> <picture.push [first piece]> <! part of the picture appears !> <picture.push [second piece]> <picture.commit> <! now the picture is complete !>
The console's icon field sets the small picture that represents your program, for example in a task bar. It takes a picture just like an img does:
print('<console.icon ', end='')
print_raw('icon.png')
print('>')
This program shows the pictures in a folder one at a time, with buttons to move between them. The console only has one img; the program replaces its picture each time the user clicks.
import os, sys
folder = 'pictures'
pictures = sorted(f for f in os.listdir(folder) if f.endswith(('.png', '.jpg', '.gif', '.svg')))
current = 0
def print_raw(filename):
with open(filename, 'rb') as f:
data = f.read()
print('[' + str(len(data)) + '{', end='', flush=True)
sys.stdout.buffer.write(data)
sys.stdout.buffer.flush()
print('}]', end='', flush=True)
def show(n):
print('<picture ', end='')
print_raw(os.path.join(folder, pictures[n]))
print('>')
print('<caption {' + pictures[n] + ' (' + str(n + 1) + ' of ' + str(len(pictures)) + ')}>')
print('''
<console.title {Picture Viewer}>
<box width=400 height=300 border=1 align={center},{middle} {<picture:img {}>}></n>
<caption:span {}></n>
<button {Previous} onClick=<putLn {/prev}>> <button {Next} onClick=<putLn {/next}>> <button {Quit} onClick=<putLn {/quit}>>
''')
show(current)
while True:
command = input()
if command == '/quit':
break
elif command == '/next':
current = (current + 1) % len(pictures)
elif command == '/prev':
current = (current - 1) % len(pictures)
show(current)
★The % in Python wraps the picture number round, so clicking Next on the last picture goes back to the first. (The file names are shown in the caption as text, which is fine for ordinary names. If a name could contain { or <, send it as raw data instead, as in the chapter on instance names.)
The audio and video tags play sound and video files. As with pictures, the primary field holds the file's data. Because there are several media formats, tell the console which one you're sending with the mimeType field:
| Format | mimeType |
|---|---|
| MP3 sound | {audio/mpeg} |
| WAV sound | {audio/wav} |
| OGG sound | {audio/ogg} |
| MP4 video | {video/mp4} |
| WebM video | {video/webm} |
print('<ding:audio visible=false mimeType={audio/wav} playing=false ', end='')
print_raw('ding.wav')
print('>')
Nothing plays until you set playing to true. So a good pattern is to load your sounds once, at the start of your program, and then play them whenever they're needed:
print('<ding.playing true>')
An audio tag has nothing to show on screen unless you set controls=true, which gives
it a small player with play, pause and volume controls. The example above also sets visible=false, so
the sound stays hidden even if controls are switched on later.
| Field | What it does |
|---|---|
| playing | true to play, false to pause |
| position | the current point in the media, in seconds; set it to jump somewhere else |
| loop | true to start again from the beginning each time it finishes |
| duration | how long the media is, in seconds (only once it's ready; see below) |
| controls | true to show play, pause and volume controls |
| Field | Happens when... |
|---|---|
| onReady | the media has loaded enough to play, and its duration is known |
| onPlay | playback starts |
| onPause | playback is paused |
| onEnded | playback reaches the end |
| onError | the media can't be played |
For example, this tells the program when a video has finished, so it can move on to whatever comes next:
print('<movie:video width=320 height=240 controls=true mimeType={video/mp4} playing=false '
'onEnded=<putLn {/movie-ended}> ', end='')
print_raw('intro.mp4')
print('>')
print('<movie.playing true>')
You don't have to wait for a whole video to arrive before it can play. When a video or sound is sent as raw data in one go, playback can begin while the rest of the data is still on its way, and onReady can come early. If you send it in pieces with .push, though, the console can't tell how big the file will be, so playback can only begin after .commit.
Media files can be large, and everything you send has to fit in the console's memory. Keep sound effects short, and videos small. If you need something long, split it into short pieces, and load the next piece while the current one plays.