Editing UBISS2024

Jump to navigation Jump to search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
 
This course is designed as a week-long tutorial to engage with ubiquitous devices in the domain of smart environments and how to use machine learning to build smart devices. Here, we use an [[Arduino_Nano_RP2040_Connect|Arduino Nano RP2040 Connect]].
 
This course is designed as a week-long tutorial to engage with ubiquitous devices in the domain of smart environments and how to use machine learning to build smart devices. Here, we use an [[Arduino_Nano_RP2040_Connect|Arduino Nano RP2040 Connect]].
 
= Final Projects =
 
 
== StressLess Shell ==
 
See the teaser video on [https://www.youtube.com/watch?v=NBSCIGqiXqM YouTube] by Songyan Teng, Jingyao Zheng, and Tim Zindulka.
 
 
== Plant Monitoring and Warning System ==
 
See the teaser video on [https://www.youtube.com/watch?v=DxjguBbUobk YouTube] by Shenxiu Wu, and Huong Nguyen.
 
 
== IntelliPen ==
 
The pen that can recognize the characters that you write! See the teaser video on [https://www.youtube.com/watch?v=WdLBq__ORBQ YouTube] by Mohammed Khalili, and Ali Mahmoudi.
 
 
== Hand Gesture Recognition ==
 
Find the code and documentation at [https://github.com/mamadzebal/Morse-Code-Detector GitHub]. The project was completed by Mohammed Farhoudi and Samira Kamali Poorazad.
 
  
 
= Schedule =
 
= Schedule =
 
== Day 1 ==
 
== Day 1 ==
* 16:00-17:00 Lecture: Introduction & Creating Interactive Smart Objects and Environments
+
* 16:00-16:15 Lecture: Introduction
* 17:00-17:45 Hands-On: [[UBISS2024#Task 1: Getting Started|Task 1: Getting Started]] Components, tools, and development environments
+
* 16:15-17:15 Lecture: Creating Interactive Smart Objects and Environments
 +
* 17:15-17:45 Hands-On: [[UBISS2024#Task 1: Getting Started|Task 1: Getting Started]] Components, tools, and development environments
 
* 17:45-18:00 Lecture: Preview of the Tasks Ahead
 
* 17:45-18:00 Lecture: Preview of the Tasks Ahead
  
 
== Day 2 ==
 
== Day 2 ==
* 10:00-10:45 Lecture: Desiging and Implementing Sensor-Based Systems
+
* 10:00-10:45 Lecture: ML Development Cycle: data collection, data cleaning, and labeling, selection of the AI/ML approach, hyper-parameter * selection and implementing the model, training the model/system, deploying the model, operations, re-training/continuous improvement  
* 10:45-11:00 ML Development Cycle: data collection, data cleaning, and labeling, selection of the AI/ML approach, hyper-parameter * selection and implementing the model, training the model/system, deploying the model, operations, re-training/continuous improvement  
+
* 10:45-11:00 Lecture: Challenges of AI/ML on Edge Devices and IoT
 
* 11:00-12:00 Hands-On: [[UBISS2024#Task 2: Read Data|Task 2: Read Data]] (accelerometer and analog pin)  
 
* 11:00-12:00 Hands-On: [[UBISS2024#Task 2: Read Data|Task 2: Read Data]] (accelerometer and analog pin)  
 
* 12:00-13:00 lunch break
 
* 12:00-13:00 lunch break
* 13:00-14:00 Hands-On: Task 3: Record the accelerometer data for four different actions (labeled dataset), store it, and transfer it to PC using the Arduino IDE
+
* 13:00-14:00 Hands-On: Task 3: read data from the accelerometer for 4 different actions, store it, transfer to PC (Arduino IDE)
 
* 14:00-14:30 Lecture: Rule-based Systems: how to design them, pros: explainability, cons: it is hard
 
* 14:00-14:30 Lecture: Rule-based Systems: how to design them, pros: explainability, cons: it is hard
* 14:30-15:00 Hands-On: Task 4: analyze the data in Excel/Google sheets and find rules for the 4 actions
+
* 14:30-15:30 Hands-On: Task 4: analyze the data in Excel/Google sheets and find rules for the 4 actions
* 15:00-15:45 Hands-On: Task 5: Implement your rule-based algorithm, optional include explanations of why the state is recognized using the Arduino IDE
+
* 15:30-16:00 Coffee break
* 15:45-16:00 get Coffee / break
+
* 16:00-16:30 Demo session 1: present your solutions
* 16:00-17:00 Ideation and testing ideas
+
* 16:30-17:45 Hands-On: Task 5: Implement your rule-based algorithm, optional include explanations of why the state is recognized (Arduino IDE)
* 17:00-17:15 Present your ideas
+
* 17:45-18:00 Hands-On: Presentations of selected results from Task 5
* 17:15-18:00 Hands-On: Discussion and presenting the results of Task 4 and 5
 
  
 
== Day 3 ==
 
== Day 3 ==
* 10:00-10:30 Lecture: Introduction to Jupyter Notebooks, training an ML model based on a given data and the self-recorded data set on the PC
+
* 10:00-12:00 Lecture: Introduction to Jupyter Notebooks, training an ML model based on a given data and the self-recorded data set on the PC (* using Google Python Notebooks or personal installation)
* 10:30-11:00 Lecture: Introduction to ML Libraries (everywhereML)
 
* 11:00-12:00 Hands-On: Project specification, Ideation on Project Ideas; and discussion of project ideas, group forming (groups of 2 or 3), Make your groups, specify your projects, see if you get the components (refine to work with the components available)
 
 
* 12:00-13:00 Lunch break
 
* 12:00-13:00 Lunch break
* 13:00-15:30 Hands-On: [[UBISS2024#Task 6: Getting Started with Jupyter Notebook|Task 6: Getting Started with Jupyter Notebook]] Installing Jupyter Notebook for Micropython, controlling LED, reading data, storing data  
+
* 13:00-15:00 Hands-On: [[UBISS2024#Task 6: Getting Started with Jupyter Notebook|Task 6: Getting Started with Jupyter Notebook]] Installing Jupyter Notebook for Micropython, controlling LED, reading data, storing data  
 +
* 15:00-15:30 Hands-On: Project specification, Ideation on Project Ideas
 
* 15:30-16:00 Coffee break
 
* 15:30-16:00 Coffee break
* 16:00-16:30 Hands-On: Presentation: status update on your project  
+
* 16:00-16:30 Hands-On: Presentation and discussion of project ideas, group forming (groups of 2 or 3)
* 16:15-18:00 Hands-On: [[UBISS2024#Task 7: Deploy Machine Learning Models|Task 7: Deploy Machine Learning Models]] Implementing a basic model using everywhereML
+
* 16:30-17:00 Lecture: Introduction to ML Libraries (everywhereML)
 +
* 17:00-18:00 Hands-On: [[UBISS2024#Task 7: Deploy Machine Learning Models|Task 7: Deploy Machine Learning Models]] Implementing a basic model using everywhereML
  
 
== Day 4 ==
 
== Day 4 ==
Line 53: Line 39:
 
* 15:00-15:30 Hands-On: stand-up meeting on project progress
 
* 15:00-15:30 Hands-On: stand-up meeting on project progress
 
* 15:30-16:00 Coffee break
 
* 15:30-16:00 Coffee break
* 16:00-16:15 Lecture: How to run your system of a battery (see [[Tutorial AutoRun]]).
+
* 16:00-17:30 Hands-On: project work
* 16:15-17:30 Hands-On: project work
 
 
* 17:30-18:00 Lecture: How to Evaluate ML Solutions (talk and discussion)
 
* 17:30-18:00 Lecture: How to Evaluate ML Solutions (talk and discussion)
  
 
== Day 5 ==
 
== Day 5 ==
 
* 10:00-10:30 Hands-On: stand-up meeting — project challenges and solutions
 
* 10:00-10:30 Hands-On: stand-up meeting — project challenges and solutions
* 10:30-11:30 Hands-On: project work and preparing the presentation
+
* 10:30-11:30 Hands-On: project work
** [[UBISS2024#Requirements_for_the_Final_Presentation | Requirements for the Presentation]]
 
 
 
 
* 11:30-12:00 Lecture: Pitfalls and Challenges in Developing ML/AI for IoT
 
* 11:30-12:00 Lecture: Pitfalls and Challenges in Developing ML/AI for IoT
 
* 12:00-13:00 Lunch break
 
* 12:00-13:00 Lunch break
Line 72: Line 55:
 
== Day 6 ==
 
== Day 6 ==
 
* 13:15-18:15: Workshop Result Presentations
 
* 13:15-18:15: Workshop Result Presentations
* 18:30-18:50: Debriefing
+
* 18:30-18:50: Debriefing in workshops
 
 
== Requirements for the Final Presentation ==
 
* The presentation has to be 4 minutes long (we stop you after 4 minutes!)
 
* First slide: Your team name and your names - and if you want a photo of the team
 
* A short video of the tech you envision (up to 60 sec, [https://www.kickstarter.com/ Kickstarter]-style promotion type)
 
* A technology description, including the list of components used in the prototype
 
* A description of your data set and how it was acquired
 
* The ML model/approach you took to learning the data
 
* An evaluation of how well your ML model works with the data set (and optional in real live)
 
 
 
== Final Submissions ==
 
You have to upload your final submission to the drive. This should include:
 
* a video where you explain your technology
 
** show the electronics components and name them
 
** show the physical setup that you created
 
** show the code you wrote and briefly explain it
 
* a zip file with all the code that is used in your project
 
* a schematic / drawing of your system as PDF or image (drawing it on paper and making a photo is fine)
 
* your final presentation (as PDF, Powerpoint)
 
* [optional] a drawing of your system architecture (hand drawing is fine)
 
  
 
= Tasks =
 
= Tasks =
Line 98: Line 61:
 
== Task 1: Getting Started ==
 
== Task 1: Getting Started ==
 
* Connect an Arduino Nano RP2040 Connect board, for this see [[Arduino_Nano_RP2040_Connect#Install the Arduino Nano RP2040 Connect Firmware|Install the Arduino Nano RP2040 Connect Firmware]]
 
* Connect an Arduino Nano RP2040 Connect board, for this see [[Arduino_Nano_RP2040_Connect#Install the Arduino Nano RP2040 Connect Firmware|Install the Arduino Nano RP2040 Connect Firmware]]
* if you use Linux/MacOS and there are issues, look for serial line permissions, e.g. https://www.xanthium.in/linux-serial-port-programming-using-python-pyserial-and-arduino-avr-pic-microcontroller and https://github.com/arduino/lab-micropython-editor/issues/64
 
* Install the Arduino Lab for MicroPython development environment, https://labs.arduino.cc/en/labs/micropython
 
 
* Task 1.1: Make the orange LED (pin 6) blink using micro python https://docs.arduino.cc/micropython/basics/digital-analog-pins/   
 
* Task 1.1: Make the orange LED (pin 6) blink using micro python https://docs.arduino.cc/micropython/basics/digital-analog-pins/   
* Task 1.2: Connect an external RGB LED (pin D2 = GPIO25, D3 = GPIO15, D4 = GPIO16) and control it (on, off, mix color, brightness), https://www.sketching-with-hardware.org/wiki/RGB_LED
+
* Task 1.2: Connect an external RGB LED (pin 2, 3, 4) and control it (on, off, mix color, brightness), https://www.sketching-with-hardware.org/wiki/RGB_LED
* Task 1.3: Add the photo resistors to your board, read their values, and write them to the file; see the instructions for [[LDR]].
 
* Task 1.4: Combine your [[LDR]] and the [[RGB_LED]] example to change the blinking interval with the light value measures.
 
  
 
=== Solution Task 1.1: LED Blinking ===
 
=== Solution Task 1.1: LED Blinking ===
Line 130: Line 89:
 
from machine import Pin
 
from machine import Pin
  
# RGB LED connected to the RP2040
+
# RGB LED connected to Nano WiFi module.
ledG = Pin(25, Pin.OUT)
+
ledG = Pin(2, Pin.OUT)
ledR = Pin(15, Pin.OUT)
+
ledR = Pin(3, Pin.OUT)
ledB = Pin(16, Pin.OUT)
+
ledB = Pin(4, Pin.OUT)
 
print("start")
 
print("start")
  
Line 150: Line 109:
 
     ledB.on()
 
     ledB.on()
 
     time.sleep_ms(250)
 
     time.sleep_ms(250)
</syntaxhighlight>
 
 
=== Solution Task 1.3 Read Light-Dependent Resistor (LDR) ===
 
See [[LDR]]. A0 is the analog input with 16 bit resolution. It reads the analog value every second and print it to the console-
 
 
<syntaxhighlight lang="python" line='line'>
 
#Example usage for Arduino Nano
 
from machine import Pin, ADC
 
from time import sleep
 
 
analogPin = ADC(Pin(26))
 
 
while True:
 
  analogVal16 = analogPin.read_u16()
 
  print(analogVal16)
 
  sleep(1)
 
</syntaxhighlight>
 
 
 
=== Solution Task 1.4 Combine Light-Dependent Resistor (LDR) with Blinking LED ===
 
<syntaxhighlight lang="python" line='line'>
 
from machine import Pin, ADC
 
import time
 
 
led = Pin(6, Pin.OUT)
 
analogPin = ADC(Pin(26))
 
 
while (True):
 
  analogVal16 = analogPin.read_u16()
 
  print(analogVal16)
 
  rate = analogVal16 / 300 # create a simple mapping
 
  led.on()
 
  time.sleep_ms(int(rate)) # convert the rate to an integer type
 
  led.off()
 
  time.sleep_ms(int(rate))
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
== Task 2: Read Data ==
 
== Task 2: Read Data ==
 
* read data from the accelerometer and the gyro and print them (Arduino IDE) https://docs.arduino.cc/micropython/basics/board-examples/
 
* read data from the accelerometer and the gyro and print them (Arduino IDE) https://docs.arduino.cc/micropython/basics/board-examples/
* extend your program to write the data from the accelerometers to a file; see the instructions for [[FileIO]].
+
* extend you program to write the data from the accelerometers to a file, https://www.sketching-with-hardware.org/wiki/FileIO
 
* transfer the file to your computer
 
* transfer the file to your computer
* optional: add the photo resistors to your board, read their values, and write them to the file; see the instructions for [[LDR]].
+
* optional: add the photo resistors to your board, read their values, and write them to the file, too, https://www.sketching-with-hardware.org/wiki/LDR
  
 
=== Solution Task 2.1: Read Accelerometer and Gyro ===
 
=== Solution Task 2.1: Read Accelerometer and Gyro ===
Line 214: Line 138:
  
 
<syntaxhighlight lang="python" line='line'>
 
<syntaxhighlight lang="python" line='line'>
 +
#Example usage for Arduino Nano
 
from machine import Pin, ADC
 
from machine import Pin, ADC
 
from time import sleep
 
from time import sleep
Line 226: Line 151:
  
 
== Task 6: Getting Started with Jupyter Notebook ==
 
== Task 6: Getting Started with Jupyter Notebook ==
* Connect the board
+
* connect the board
* Install the [[Jupyter Notebook]],  
+
* install the Juypter Notebook, https://www.sketching-with-hardware.org/wiki/Jupyter
* Read the accelerometer and the gyro and show it in the notebook
+
* read the accelerometer and the gyro and show it in the notebook
 +
 
  
 
=== Task 6.1: is it moved? ===
 
=== Task 6.1: is it moved? ===
Line 245: Line 171:
 
== Task 7: Deploy Machine Learning Models ==
 
== Task 7: Deploy Machine Learning Models ==
 
We will use https://github.com/eloquentarduino/everywhereml to detect the same gestures as in Task 2.2. For this, install everywhereml:
 
We will use https://github.com/eloquentarduino/everywhereml to detect the same gestures as in Task 2.2. For this, install everywhereml:
 
See [[EverywhereML]] for downloading the full example and a dataset to experiment with.
 
 
 
<syntaxhighlight lang="Bash">
 
<syntaxhighlight lang="Bash">
pip3 install -U everywhereml
+
pip3 install -U everywhere
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 268: Line 191:
 
== Task 8: Connect to WiFi ==
 
== Task 8: Connect to WiFi ==
 
This is an optional task.
 
This is an optional task.
* Connect both boards to WIFI using [[Tutorial Network]]
+
* connect both boards to WIFI using [[Tutorial_Network]]
* Use the Arduino Nano RP2040 Connect as output (showing a color)
+
* use the Arduino Nano RP2040 Connect as output (showing a color)
* Use the Arduino Nano RP2040 Connect as input (recognize with rules 3 gestures)
+
* use the Arduino Nano RP2040 Connect as input (recognize with rules 3 gestures)
  
 
= Links =
 
= Links =
Line 321: Line 244:
 
C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook
 
C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook
  
[[Category:Courses]]
+
[[Category:Course]]
 
[[Category:UBISS2024]]
 
[[Category:UBISS2024]]
 
[[Category:Arduino Nano RP2040 Connect]]
 
[[Category:Arduino Nano RP2040 Connect]]
[[Category:MicroPython]]
 

Please note that all contributions to Sketching with Hardware at LMU Wiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see My wiki:Copyrights for details). Do not submit copyrighted work without permission!

Cancel Editing help (opens in new window)