UBISS2024

From Sketching with Hardware at LMU Wiki
Jump to navigation Jump to search

Tasks

Project 0: connect a Arduino Nano ESP32 board

solution Project 0: LED Blinking

 1 # Blinky example
 2 
 3 import time
 4 from machine import Pin
 5 
 6 # This is the only LED pin available on the Nano RP2040,
 7 # other than the RGB LED connected to Nano WiFi module.
 8 led = Pin(6, Pin.OUT)
 9 
10 while (True):
11    led.on()
12    time.sleep_ms(250)
13    led.off()
14    time.sleep_ms(200)

solution Project 0: Control external RGB

 1 # RGB example
 2 
 3 import time
 4 from machine import Pin
 5 
 6 # RGB LED connected to Nano WiFi module.
 7 ledG = Pin(2, Pin.OUT)
 8 ledR = Pin(3, Pin.OUT)
 9 ledB = Pin(4, Pin.OUT)
10 print("start")
11 
12 while (True):
13     print("*")
14     ledG.on()
15     ledR.off()
16     ledB.off()
17     time.sleep_ms(250)
18     ledG.off()
19     ledR.on()
20     ledB.off()
21     time.sleep_ms(250)
22     ledG.off()
23     ledR.off()
24     ledB.on()
25     time.sleep_ms(250)

Project 1: read Acceleration from Arduino Nano ESP32 board

solution Project 1: Read Accelerometer and Gyro

 1 import time
 2 from lsm6dsox import LSM6DSOX
 3 
 4 from machine import Pin, I2C
 5 lsm = LSM6DSOX(I2C(0, scl=Pin(13), sda=Pin(12)))
 6 
 7 while (True):
 8     accel_data = lsm.accel()
 9     print('Accelerometer: x:{:>8.3f} y:{:>8.3f} z:{:>8.3f}'.format(*accel_data))
10     gyro_data = lsm.gyro()
11     print('Gyroscope:     x:{:>8.3f} y:{:>8.3f} z:{:>8.3f}'.format(*gyro_data))
12     print("")
13     time.sleep_ms(100)


solution Project 2: Read analog values

Code Example Arduino Nano Connect RP2040

A0 is the analog input with 16 bit resolution. It reads the analog value every second and print it to the console- 

 1 #Example usage for Arduino Nano
 2 from machine import Pin, ADC
 3 from time import sleep
 4 
 5 analogPin = ADC(Pin(26))
 6 
 7 while True:
 8   analogVal16 = analogPin.read_u16()
 9   print(analogVal16)
10   sleep(1)

Project 2: Jupyter Notebook


Task 2.1: is it moved?

  • read acceleration and gyro
  • calculate the differences between values
  • show an ouput when it is move
  • create a file on the device that logs, when it is moved

Task 2.2: it was turned upside down?

  • read acceleration and gyro
  • make a rule based "AI" that records
    • it was put upside down
    • it was turned 360
    • it was moved "quickly"

Task 3: ML on Arduino Nano Connect RP2040

Task 4: connect both boards to WIFI

  • connect both boards to WIFI using Tutorial_Network
  • use the Arduino Nano ESP32 as output (showing a color)
  • use the Arduino Nano Connect RP2040 as input (recognize with rules 3 gestures)

Links

Micropython Basics

https://docs.arduino.cc/micropython/#micropython-101 https://docs.arduino.cc/micropython/basics/board-examples/ https://www.codemotion.com/magazine/backend/getting-started-with-micropython-on-arduino-nano-rp2040-connect/ https://www.penguintutor.com/programming/arduino-python https://micropython.org/ https://docs.arduino.cc/micropython/micropython-course/course/installation/ https://docs.arduino.cc/micropython/micropython-course/course/examples/ https://wellys.com/posts/rp2040_micropython_1/ https://micropython.org/download/RPI_PICO_W/

Python / Jupyter Notebooks for Hardware

https://www.sketching-with-hardware.org/wiki/Jupyter https://towardsdatascience.com/micropython-on-esp-using-jupyter-6f366ff5ed9 https://www.datacamp.com/tutorial/markdown-in-jupyter-notebook https://saturncloud.io/blog/how-to-import-python-file-as-module-in-jupyter-notebook/ https://jupyter.org/install https://www.geeksforgeeks.org/install-jupyter-notebook-in-windows https://www.instructables.com/Micropython-on-ESP-Using-Jupyter/

Development environments

https://labs.arduino.cc/en/labs/micropython https://labs.arduino.cc/en/labs/micropython-installer https://www.arduino.cc/en/software

Libraries

https://github.com/jposada202020/MicroPython_LSM6DSOX

Data sheets and resources

https://micropython.org/download/ARDUINO_NANO_RP2040_CONNECT/ https://docs.arduino.cc/resources/pinouts/ABX00083-full-pinout.pdf


Tutorials

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-openmv-setup/ https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-data-logger/

Machine Learning Basics

https://github.com/eloquentarduino/everywhereml https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html https://www.linkedin.com/pulse/arduino-truly-tiny-machine-learning-simone-salerno https://eloquentarduino.com/posts/micropython-machine-learning https://github.com/mocleiri/tensorflow-micropython-examples https://dev.to/tkeyo/tinyml-machine-learning-on-esp32-with-micropython-38a6

Networking Basics

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-ap-web-server-rgb/ https://docs.micropython.org/en/latest/library/socket.html

Local Links

https://ubicomp.net/sw/db1/var2db.php? http://localhost:8888/notebooks/ArduinoNanoRP2040_v01.ipynb http://localhost:8888/doc/tree/create-ML-model01.ipynb

Reading

Required Reading before the course

Albrecht Schmidt. 2020. Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 3, 1–4. https://doi.org/10.1145/3399715.3400873 (4p) https://uni.ubicomp.net/as/iHCAI2020.pdf

A. Schmidt and K. van Laerhoven, "How to build smart appliances?," in IEEE Personal Communications, vol. 8, no. 4, pp. 66-71, Aug. 2001, doi: 10.1109/98.944006. (6p) https://www.eti.uni-siegen.de/ubicomp/papers/sl_ieeepc2001.pdf

Schmidt, Albrecht. "Understanding and researching through making: a plea for functional prototypes." interactions 24.3 (2017): 78-81. (4p) https://www.sketching-with-hardware.org/files/functional3058498.pdf

Huy Viet Le, Sven Mayer, and Niels Henze. 2020. Deep learning for human-computer interaction. interactions 28, 1 (January - February 2021), 78–82. (5p) https://doi.org/10.1145/3436958 https://sven-mayer.com/wp-content/uploads/2021/01/huy2021deep.pdf

Le, Huy Viet; Mayer, Sven; Weiß, Max; Vogelsang, Jonas; Weingärtner, Henrike; Henze, Niels (2020) Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones. In: ACM Trans. Comput.-Hum. Interact., vol. 27, no. 5, pp. 38, 2020, ISSN: 1073-0516. (38p) https://sven-mayer.com/wp-content/uploads/2020/09/le2020shortcuts.pdf

Hurwitz, Judith, and Daniel Kirsch. "Machine learning for dummies." IBM Limited Edition 75 (2018): 9780429196645-6. https://www.ibm.com/downloads/cas/GB8ZMQZ3 (Pages 3-18 and 29-47, this is Chapters 1 and 3) (35p)

Chris Garrett. MicroPython: An Intro to Programming Hardware in Python https://realpython.com/micropython/ (14 pages)

MicroPython Basics https://docs.arduino.cc/micropython/basics/micropython-basics/ (5 pages)

Recommended Reading before the course:

John D. Kelleher, Deep Learning, https://mitpress.mit.edu/9780262537551/deep-learning/

Yuli Vasiliev, Python for Data Science: A Hands-On Introduction, https://nostarch.com/python-data-science

Tutorial on Jupyter Notebooks: https://www.datacamp.com/tutorial/tutorial-jupyter-notebook

Smola, Alex, and S. V. N. Vishwanathan. "Introduction to machine learning." Cambridge University, UK 32.34 (2008): 2008. https://alex.smola.org/drafts/thebook.pdf


Random Commands

pip install micropython-lsm6dsox

picotool.exe load -x C:\Users\ru42qak\AppData\Roaming\OpenMV\openmvide\firmware\ARDUINO_NANO_RP2040_CONNECT\firmware.bin

pip install jupyterlab

pip install everywhereml

python -m pip install jupyter

git clone https://github.com/goatchurchprime/jupyter_micropython_kernel.git

pip install -e jupyter_micropython_kernel

python -m notebook

python -m jupyter kernelspec list


C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>pip install -e jupyter_micropython_kernel

C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook

Retrieved from "http://www.sketching-with-hardware.org/wiki/index.php?title=UBISS2024&oldid=1305"