UBISS2024

From Sketching with Hardware at LMU Wiki
Jump to navigation Jump to search

Tasks

Project 0: connect a Arduino Nano ESP32 board

solution Project 0: LED Blinking

 1 # Blinky example
 2 
 3 import time
 4 from machine import Pin
 5 
 6 # This is the only LED pin available on the Nano RP2040,
 7 # other than the RGB LED connected to Nano WiFi module.
 8 led = Pin(6, Pin.OUT)
 9 
10 while (True):
11    led.on()
12    time.sleep_ms(250)
13    led.off()
14    time.sleep_ms(200)

solution Project 0: Control external RGB

 1 # RGB example
 2 
 3 import time
 4 from machine import Pin
 5 
 6 # RGB LED connected to Nano WiFi module.
 7 ledG = Pin(2, Pin.OUT)
 8 ledR = Pin(3, Pin.OUT)
 9 ledB = Pin(4, Pin.OUT)
10 print("start")
11 
12 while (True):
13     print("*")
14     ledG.on()
15     ledR.off()
16     ledB.off()
17     time.sleep_ms(250)
18     ledG.off()
19     ledR.on()
20     ledB.off()
21     time.sleep_ms(250)
22     ledG.off()
23     ledR.off()
24     ledB.on()
25     time.sleep_ms(250)

Project 1: read Acceleration from Arduino Nano ESP32 board

solution Project 1: Read Accelerometer and Gyro

 1 import time
 2 from lsm6dsox import LSM6DSOX
 3 
 4 from machine import Pin, I2C
 5 lsm = LSM6DSOX(I2C(0, scl=Pin(13), sda=Pin(12)))
 6 
 7 while (True):
 8     accel_data = lsm.accel()
 9     print('Accelerometer: x:{:>8.3f} y:{:>8.3f} z:{:>8.3f}'.format(*accel_data))
10     gyro_data = lsm.gyro()
11     print('Gyroscope:     x:{:>8.3f} y:{:>8.3f} z:{:>8.3f}'.format(*gyro_data))
12     print("")
13     time.sleep_ms(100)

Project 2: Jupyter Notebook


Task 2.1: is it moved?

  • read acceleration and gyro
  • calculate the differences between values
  • show an ouput when it is move
  • create a file on the device that logs, when it is moved

Task 2.2: it was turned upside down?

  • read acceleration and gyro
  • make a rule based "AI" that records
    • it was put upside down
    • it was turned 360
    • it was moved "quickly"

Task 3: ML on Arduino Nano Connect RP2040

Task 4: connect both boards to WIFI

  • connect both boards to WIFI using Tutorial_Network
  • use the Arduino Nano ESP32 as output (showing a color)
  • use the Arduino Nano Connect RP2040 as input (recognize with rules 3 gestures)

Links

Micropython Basics

https://docs.arduino.cc/micropython/#micropython-101 https://docs.arduino.cc/micropython/basics/board-examples/ https://www.codemotion.com/magazine/backend/getting-started-with-micropython-on-arduino-nano-rp2040-connect/ https://www.penguintutor.com/programming/arduino-python https://micropython.org/ https://docs.arduino.cc/micropython/micropython-course/course/installation/ https://docs.arduino.cc/micropython/micropython-course/course/examples/ https://wellys.com/posts/rp2040_micropython_1/ https://micropython.org/download/RPI_PICO_W/

Python / Jupyter Notebooks for Hardware

https://www.sketching-with-hardware.org/wiki/Jupyter https://towardsdatascience.com/micropython-on-esp-using-jupyter-6f366ff5ed9 https://www.datacamp.com/tutorial/markdown-in-jupyter-notebook https://saturncloud.io/blog/how-to-import-python-file-as-module-in-jupyter-notebook/ https://jupyter.org/install https://www.geeksforgeeks.org/install-jupyter-notebook-in-windows https://www.instructables.com/Micropython-on-ESP-Using-Jupyter/

Development environments

https://labs.arduino.cc/en/labs/micropython https://labs.arduino.cc/en/labs/micropython-installer https://www.arduino.cc/en/software

Libraries

https://github.com/jposada202020/MicroPython_LSM6DSOX

Data sheets and resources

https://micropython.org/download/ARDUINO_NANO_RP2040_CONNECT/ https://docs.arduino.cc/resources/pinouts/ABX00083-full-pinout.pdf


Tutorials

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-openmv-setup/ https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-data-logger/

Machine Learning Basics

https://github.com/eloquentarduino/everywhereml https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html https://www.linkedin.com/pulse/arduino-truly-tiny-machine-learning-simone-salerno https://eloquentarduino.com/posts/micropython-machine-learning https://github.com/mocleiri/tensorflow-micropython-examples https://dev.to/tkeyo/tinyml-machine-learning-on-esp32-with-micropython-38a6

Networking Basics

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-ap-web-server-rgb/ https://docs.micropython.org/en/latest/library/socket.html

Local Links

https://ubicomp.net/sw/db1/var2db.php? http://localhost:8888/notebooks/ArduinoNanoRP2040_v01.ipynb http://localhost:8888/doc/tree/create-ML-model01.ipynb

Reading

Required Reading before the course

Albrecht Schmidt. 2020. Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 3, 1–4. https://doi.org/10.1145/3399715.3400873 (4p) https://uni.ubicomp.net/as/iHCAI2020.pdf

A. Schmidt and K. van Laerhoven, "How to build smart appliances?," in IEEE Personal Communications, vol. 8, no. 4, pp. 66-71, Aug. 2001, doi: 10.1109/98.944006. (6p) https://www.eti.uni-siegen.de/ubicomp/papers/sl_ieeepc2001.pdf

Schmidt, Albrecht. "Understanding and researching through making: a plea for functional prototypes." interactions 24.3 (2017): 78-81. (4p) https://www.sketching-with-hardware.org/files/functional3058498.pdf

Huy Viet Le, Sven Mayer, and Niels Henze. 2020. Deep learning for human-computer interaction. interactions 28, 1 (January - February 2021), 78–82. (5p) https://doi.org/10.1145/3436958 https://sven-mayer.com/wp-content/uploads/2021/01/huy2021deep.pdf

Le, Huy Viet; Mayer, Sven; Weiß, Max; Vogelsang, Jonas; Weingärtner, Henrike; Henze, Niels (2020) Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones. In: ACM Trans. Comput.-Hum. Interact., vol. 27, no. 5, pp. 38, 2020, ISSN: 1073-0516. (38p) https://sven-mayer.com/wp-content/uploads/2020/09/le2020shortcuts.pdf

Hurwitz, Judith, and Daniel Kirsch. "Machine learning for dummies." IBM Limited Edition 75 (2018): 9780429196645-6. https://www.ibm.com/downloads/cas/GB8ZMQZ3 (Pages 3-18 and 29-47, this is Chapters 1 and 3) (35p)

Chris Garrett. MicroPython: An Intro to Programming Hardware in Python https://realpython.com/micropython/ (14 pages)

MicroPython Basics https://docs.arduino.cc/micropython/basics/micropython-basics/ (5 pages)

Recommended Reading before the course:

John D. Kelleher, Deep Learning, https://mitpress.mit.edu/9780262537551/deep-learning/

Yuli Vasiliev, Python for Data Science: A Hands-On Introduction, https://nostarch.com/python-data-science

Tutorial on Jupyter Notebooks: https://www.datacamp.com/tutorial/tutorial-jupyter-notebook

Smola, Alex, and S. V. N. Vishwanathan. "Introduction to machine learning." Cambridge University, UK 32.34 (2008): 2008. https://alex.smola.org/drafts/thebook.pdf


Random Commands

pip install micropython-lsm6dsox

picotool.exe load -x C:\Users\ru42qak\AppData\Roaming\OpenMV\openmvide\firmware\ARDUINO_NANO_RP2040_CONNECT\firmware.bin

pip install jupyterlab

pip install everywhereml

python -m pip install jupyter

git clone https://github.com/goatchurchprime/jupyter_micropython_kernel.git

pip install -e jupyter_micropython_kernel

python -m notebook

python -m jupyter kernelspec list


C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>pip install -e jupyter_micropython_kernel

C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook

Retrieved from "http://www.sketching-with-hardware.org/wiki/index.php?title=UBISS2024&oldid=1300"