Contents
- 1 Quick Start Pack
- 2 Download All Source
- 3 Pick & Choose: Inputs
- 3.1 Arduino
- 3.2 Audio
- 3.3 BBC micro:bit
- 3.4 Bitalino (biosignals)
- 3.5 emakefun nano kit
- 3.6 GameTrak “Real World Golf” tether (game controller)
- 3.7 iOS / iPhone / iPad
- 3.8 Java
- 3.9 Leap Motion (hardware sensor)
- 3.10 Logitech Extreme 3D Pro Joystick
- 3.11 Max/MSP (audio)
- 3.12 Microsoft Kinect (3D body tracking/game controller)
- 3.13 Myo Armband (EMG/muscle sensing and movement)
- 3.14 openFrameworks (C++, animation)
- 3.15 PureData (PD) (audio)
- 3.16 Processing (animation, screen-based input)
- 3.17 Python
- 3.18 SuperCollider
- 3.19 TouchOSC for mobile devices and tablets (iOS, Android):
- 3.20 Video/Webcam
- 3.21 Wacom / Graphics tablet stylus
- 3.22 WebSockets (for desktop & mobile via browser apps)
- 3.23 WiiMote (game controller)
- 3.24 word2vec
- 3.25 XBox 360 and similar USB gamepads
- 4 Pick & Choose: Outputs
- 5 Teaching Examples
Quick Start Pack
This contains a small number of very useful executable examples for inputs, outputs, and teaching.
- Windows (32-bit, 64-bit)
- Mac
- Alternative for McAfee users: This Mac bundle does not include the ChucK outputs created with Platypus, which McAfee erroneously thinks are dangerous
- Linux (32-bit, 64-bit)
- Source code (in Processing, with some optional openFrameworks and ChucK)
- Take a look at this video demonstration of how to run examples in Processing
- Last resort: If you have trouble running the examples above, you may be able to run the examples below, which come with Java packaged in; but beware that these downloads are much bigger! Also, be aware that webcam examples may not work in some current versions of Windows and OS X (Catalina).
- Mac with embedded Java:
- Inputs: Mouse-dragged box, Video face input, Mouse input for DTW
- Outputs: FM synthesis, Simple continuous color control, Simple classifier control of color and sound, Continuously-controlled drum machine, Continuously-controlled particle system, Simple DTW-controlled game, Simple text trigger display for DTW
- Teaching examples: Classification Explorer, Regression Explorer, DTW Mouse Explorer
- Windows (32-bit) with embedded Java:
- Inputs: Mouse-dragged box, Video face input, Mouse input for DTW
- Outputs: FM synthesis, Simple continuous color control, Simple classifier control of color and sound, Continuously-controlled drum machine, Continuously-controlled particle system, Simple DTW-controlled game, Simple text trigger display for DTW
- Teaching examples: Classification Explorer, Regression Explorer, DTW Mouse Explorer
- Mac with embedded Java:
Download All Source
Want to jump right in and start with the source code? If so, you can download the very latest source for all examples from github.
Pick & Choose: Inputs
Arduino
- Using Processing or an executable to translate serial to OSC
- First, download this code to run on the Arduino
- Then, choose one of the following to translate Arduino serial data into OSC messages
- Or, you can use Max/MSP to translate serial from Arduino to OSC
Audio
- MFCCs (Mel-frequency Cepstral Coefficients)
- Source code in openFrameworks
- Executable
- Various audio features (FFT, MFCCs, Constant-Q, Peak frequency, Spectral Centroid, RMS)
- Source code in openFrameworks
- Executable
- Various features using Max/MSP
- Scott Hawley has made a Web Audio feature extractor (thanks Scott!)
BBC micro:bit
- Step 1. Download this file and save it on your microbit (e.g., in Mac Finder or Windows Explorer). It should be the only file on your microbit.
- Step 2. Download the desired version of your desktop software below:
- Microbit using Processing (any operating system; requires Processing):
- Microbit on Mac
- Microbit on Mac “last resort” (with Java included – big download)
- Microbit on Windows 32-bit, 64-bit
- Step 3. Plug in your micro:bit to your computer via USB, run the desktop software you just downloaded, and change the device ID in the top dropdown box until you see the accelerometer numbers changing.
- Step 4. In the same program, choose which input types you want to send to Wekinator using the bottom dropdown box, and look at the screen to see how many inputs you have total. Tell Wekinator to listen for this many inputs.
Bitalino (biosignals)
- Coming soon
emakefun nano kit
- First, download the drivers from https://github.com/emakefun/emakefun-nano-kit
- Second, run one of the following to get data from the device as OSC messages:
GameTrak “Real World Golf” tether (game controller)
iOS / iPhone / iPad
- Louis McCallum has made the free MotionSender app to send accelerometer, gyroscope, and attitude data to Wekinator.
- You might also check out the TouchOSC app, which is more complex to set up but allows you to use other controls (e.g., touch buttons)
Java
Leap Motion (hardware sensor)
Logitech Extreme 3D Pro Joystick
- Joystick using ChucK
- Joystick executable for Mac (coming soon)
Max/MSP (audio)
- Simple on-screen sliders
- (See other example in Audio, above)
Microsoft Kinect (3D body tracking/game controller)
- Using openFrameworks
- Executable for Mac
- C++ source code for Windows created by David Famillian
Myo Armband (EMG/muscle sensing and movement)
- You can use the Myo Mapper software to connect Myo to Wekinator. You’ll first have to install the Myo Connect software and connect your Myo to your computer.
- Alternatively, you can use myo for max within Max/MSP
- Or you can use this JS command-line tool by Jonathan Wohl
openFrameworks (C++, animation)
- Simple mouse example
- (See other examples in Audio, Kinect, Video categories)
PureData (PD) (audio)
Processing (animation, screen-based input)
- Simple mouse x,y position
- Simple mouse dragged object
- Mouse x, y position for training dynamic time warping
Python
SuperCollider
- Example input and output in single SuperCollider patch by Simon Katan (warning: uses non-default output port 57120 in Wekinator)
TouchOSC for mobile devices and tablets (iOS, Android):
- You can route TouchOSC data through Max/MSP (requires Max/MSP to be installed)
- Or you can run this OSCulator file to use accelerometer, 2D slider, and button data from TouchOSC via OSCulator (runs on Mac only)
Video/Webcam
- Simple 10×10 color grid (not useful for much)
- Track the x,y position of a coloured object using your webcam
- Max/MSP patch by Sam Parke-Wolfe
- Mac executable version of the above (big download but doesn’t require Max/MSP installation)
- Face Tracking (Processing version; tracks the x- and y- position and width of one face)
- Face Tracking (openFrameworks version; tracks the x- and y- position and width of one face)
- Source code in openFrameworks
- Mac executable Note: If this doesn’t put sunglasses on your face when you run it, drag the app to a new location (e.g., your desktop) then back. Make sure the app and the data/ directory share the same parent directory. Run it again. (This is due to a peculiarity in security settings on OS X 10.12 and higher.)
- More detailed facial feature tracking (e.g., for facial expressions), with openFrameworks
- Source code: Use the FaceOSC example in Gene Kogan’s ofxFaceTracker
- Mac executable
- Windows executable (coming soon)
- Face tracking on iOS (thanks Scott Hawley!)
- Deep learning features for training object recognisers (e.g., hand versus cup versus shoe): more sophisticated, but very slow. See README for information about how to run it.
- Source code in openFrameworks
- Mac executable Note: don’t forget to follow instructions in the README!
- Windows executable (coming soon)
Wacom / Graphics tablet stylus
WebSockets (for desktop & mobile via browser apps)
- Mouse via Web Sockets (view in github, thanks Nick Nikolov!)
- Accelerometer (i.e., for mobile phone tilt) (github)
- Another mouse example using node from noisyneuron
- P5.js integration example from Derick Ostrenko
WiiMote (game controller)
- Using OSCulator (Mac only)
word2vec
- This python script allows you to type English words on the command line. A feature vector capturing information about the linguistic context of the word (roughly related to its semantic and syntactic function) is sent to Wekinator. This feature extractor uses the word2vec word embedding using the glove-twitter-25 dataset (which is pre-trained and downloaded automatically by the python script).
XBox 360 and similar USB gamepads
- Scott Hawley has made a Processing sketch for this!
Pick & Choose: Outputs
AbletonLive (audio)
- Using openFrameworks (Thanks Gene Kogan!)
- You can also use the MIDI continuous control example below (press the CC buttons for MIDI mapping support)
ChucK (audio)
- Simple drum machine classifier
- FM Synthesis
IFTTT (“if this then that”)
Max/MSP (audio)
- Simple sine control
- Blotar physical model
- If you have a new OS, may require grabbing an updated blotar object. This will need to go into your Max path.
- Granular speed/pitch/volume control of sample
- Requires installation of MuBu and audio.ctrl.maxpat from https://github.com/bcaramiaux/Gestural-Sound-Toolkit , and requires these be in your Max path
MIDI
- Send NoteOn pitch & velocity
- Send continuous control
- Source code in Processing
- Executable:
- Win32, Win64
- Mac
- Linux32, Linux64
openFrameworks (C++, animation)
- VERY simple colour change (just adjusts hue)
- Simple 3D penguin rotation (based on ofxAssimpModelLoader example)
- Source code
- Mac executable Note: If you don’t see the penguin when you run it, drag the app to a new location (e.g., your desktop) then back. Make sure the app and the data/ directory share the same parent directory. Run it again. (This is due to a peculiarity in security settings on OS X 10.12 and higher.)
PureData (PD) (audio)
Processing (animation + audio)
- Simple continuously-controlled drum machine
- Simple continuously-controlled particle system
- FM Synthesis (sound)
- Simple DTW-controlled game
- Simple continuous color control
- Simple classifier control of color and sound
- Simple classifier probability distribution visualiser
- Simple object animation (5 continuous control parameters)
- Simple text display for classifier
- Processing source code
- Executable:
- Simple text trigger display for DTW
- Text and sound trigger display for DTW
SuperCollider
- Example input and output in single SuperCollider patch by Simon Katan (warning: uses non-default output port 57120 in Wekinator)
Unity (game engine)
- Simple object rotation Simple object rotation (Choose version for Unity 4.6 or Unity 5.6.1)
Teaching Examples
Classification Explorer
- Processing source code
- Executable:
Regression Explorer
- Processing source code
- Executable:
DTW Mouse Explorer
- Processing source code
- Executable:
OSC Data Plotter
Plots data received by OSC in real-time. Good for inspecting features and/or Wekinator output.