69 views
   owned this note
# From zero to hero with Python (History) **Date**: 17-10-2024 - 18-10-2024 **Time**: 09:00 - 15:00 (CEST) **Location**: Online ([Zoom](https://epfl.zoom.us/j/64829799035?pwd=DYKW7EWKUzurBebej4hNaJEIW9F2xn.1)) **Code of Conduct**: [The Carpentries code of conduct](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html) :::info ### Code of Conduct Participants are expected to follow those guidelines: * Use welcoming and inclusive language * Be respectful of different viewpoints and experiences * Gracefully accept constructive criticism and give constructive feedback * Focus on what is best for the community * Show courtesy and respect towards other community members ::: ### Schedule for the workshop | Day | Tutorial | Instructor | | --- | --- | --- | | 1-Thursday morning (17th Oct) | [Programming environment setup until Libraries](http://swcarpentry.github.io/python-novice-gapminder/index.html) | Silvia | | 2-Thursday afternoon (17th Oct) | [Reading Tabular to Plotting](http://swcarpentry.github.io/python-novice-gapminder/07-reading-tabular.html) | Teresa | | 3-Friday morning (18th Oct) | [Lists until Variable Scope](http://swcarpentry.github.io/python-novice-gapminder/11-lists.html) | Rabea | | 4-Friday afternoon (18th Oct) | [Programming Style and exercises](http://swcarpentry.github.io/python-novice-gapminder/18-style.html) | Till | [TOC] ### Before Introduction: who is who? Introduction to BioNT --> ### Recommendations :::info To follow the workshop more efficiently, we recommend having a two-screen setup: for example, one to display the instructor’s shared screen and the collaborative pad, and another one for your own screen ::: ### How will the workshop be run? Why a specific setup for this workshop? - We want to welcome participants for different areas and (e.g. SMEs, job seeker, academia) - Importance of privacy - We would like to create an interactive atmosphere How will we do? - Zoom with panel view - Panelist: instructors & helpers - Only trainers will be visible - No personal data will be displayed - This [HedgeDoc](https://biont.biobyte.de/fsB71mmfSGm_fY5t2gG-OA#) document in Markdown for interactions - Markdown: lightweight markup language - [Documentation](https://biont.biobyte.de/features#Edit) ### How to participate? **Ask your questions, raise issues, interact with us in this Document** In addition, to help you navigate this document, we followed the structure of the tutorial and included: - Each Hands-on section (✏️ - where you will have to work) of the tutorial, including a part to ask questions or post issues you might face :::warning ✏️ Hands-on: Topic ##### ❓ Are you finished with this section? Add a '+' below - Yes: +++++++++++ - - Waiting for the job to be done: - Need help: ##### Your questions Q: I need help with ... A: here is the answer I still do not ... ##### Do you need help? Please describe your issue - ::: An helper will help you - Question sections (❓ - where we ask you something ) for answering :::success ❓ We have some question: - ::: **Lets try the Set up together** :::warning ##### ✏️ Hands-on: Access this HedgeDoc main document ##### ❓ Are you on this HedgeDoc? (+Add a + when done) - Yes: + - No (please sent a e-mail to muellert@informatik.uni-freiburg.de) ##### ❓ Do you need help? Please describe your issue - - - - - - + ::: :::success ##### ❓ Have you ever used Markdown? (Add a +) - Yes ++++++++ - No+++++++++++++++++++++ - What is Markdown? +++++ ::: :::success ##### ❓ We plan to adjust the break times according to the workshop's progression. Does this arrangement work well for everyone? - Yes ++++++++++++++++++++++++++++++ - No ::: :::success ##### ❓ Those that answered no, could you clarify your needs? Do you need a specific break time? - - - ::: ### Icebreaker :::success ##### ❓ Tell us about a recent *First* in your life. This could be big or small, perhaps you bought a house for the first time or you tried a new restaurant in your city. Recent can be any time in the past year. - Started a Python course - I went to the night club as wheelchair user. Fun - Apartment hunting - I started therapy - Started a PhD -Went to Canary Islands - Created first poster for a conference (yet to be presented though) - Started my first job :) - started a spin-out company with university colleagues - I'm a chemist, but I'm a structural bioinformatican + - start career in neuroscience - sfssfs - Started writting a book - - I started learning guitar. - First academic publication - - Fixed old Mac Books on my own - Starting Music lessons - I tried to film a stop motion movie -Get better in gardening - - Changed windows in an apartment -+ - - Tried to fix the car on my own - - started writing poems - - Fixed house floors on my own - - - - - ::: ### Programming languages and why learning Python? :::success ##### ❓ Have you ever used a programming language? (Add a +) - Yes +++++++R+++++++++++ - No +++++++++++ ::: :::success ##### ❓ What programming languages do you know? - R - R - R - R - R - Python - html - Python - html and css - Python - R - R, C - Rust - R - bash - R - bash - a little bash - R - fortran - - R - R - - a little python - - - ::: :::success ##### ❓ Do you have Anaconda installed? (Add a +) - Yes ++++++++++++++++++++++++++++ - No + + comment from a user: Actually Anaconda is not mentioned in the e-mails sent. I recommend to include it. - We did sent out two emails about the installation. And added this [instalation instuctions](https://members.cecam.org/workshops/1362/edit-steep/6 ) on CECAM. We are sorry if you did not get this emails. Could you check your Spam settings. Please email us if there are still installation issues. #### What was your issue? Q:Not sure if I clicked to include anaconda in the PATH A: which operating system you are using ? Window, linux, MAC ? Windows >. Check instruction here :https://docs.anaconda.com/anaconda/install/windows/, hope it will help i am installing now>>> okay, let us know if you face any issue Q:I have just download the data folderyes A: Have you installed Anaconda? Then its fine, great Q: I have JupyterLab at Posit Workbench, is that okay? A: As long as you are able to run the code as like instructor, i guess it's fine Q: Anaconda is installed but the URL is blocked by our institute. Not sure if this is an issue for the course? Anaconda homepage, we are not able to update packages via this page. Ok, thank you!+ A: Which URL are you referring to? If Anaconda is installed, then no problem Q:Can you pls show the code for UBUNTU again A: You mean for installtion of Anaconda ? `bash ~/Downloads/Anaconda3-<INSTALLER_VERSION>-Linux-x86_64.sh` this is the command when you downloaded the Anaconda installer for Ubuntu and it's located at your Download folder Q:for opening jupiter A: if Anaconda installed , just type 'jupyter lab' ( without quotes) and it will open Q: A: ::: :::success ##### ❓ Do you have the gapminder data? (Add a +)+ - Yes +++++++++++++++++++++´+++++++++ - No ++ #### What was your issue? Q: A:i downloaded data and dont know how to instal. I installed anaconda Q: I cannot install Anaconda. My computer fails to extract packages A: Check you have right package extractor tool installed in your system Q: I do not know what data should I have installed. I have Anaconda installed. Ok, thanks A: you can downloade it : http://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip -The letters in jupiternotebook are italic ::: ## Day 1 morning - Thursday ### Schedule | Starting time| Duration | Content | | -------- | -------- | ---------------- | | 09:00 | 20 min | [Welcome and Setup](http://swcarpentry.github.io/python-novice-gapminder/index.html)| | 09:20 | 20 min |[Running and Quitting](http://swcarpentry.github.io/python-novice-gapminder/01-run-quit.html) | | 09:40 | 30 min |[Variables and Assignment](http://swcarpentry.github.io/python-novice-gapminder/02-variables.html)| | 10:10 | 15 min | Break | | 10:25 | 30 min | [Data Types and Type Conversion](http://swcarpentry.github.io/python-novice-gapminder/03-types-conversion.html)| | 11:00 | 30 min | [Built-in Functions and Help](http://swcarpentry.github.io/python-novice-gapminder/04-built-in.html) | | 11:30 | 30 min | [Libraries](http://swcarpentry.github.io/python-novice-gapminder/06-libraries.html) | | 12:00 | 1 h | Lunch break | :::success ##### ❓ Before we start: Is the screen clearly visible (add '+') - Yes ++++++++++++++++++++ - Please zoom in - Can you put your cursor more visible? I can't see what you click :) ++ ::: :::success ##### ❓Can you see the JupyterLab main page in your browser? Add a '+' below - Yes: +++++++++++++++++++++++ - Need time:++ - Need help: + ##### ❓ Do you need help? Please describe your issue - Can you please show us again how to open - Silvia did show it again. Hope it worked. - - Opening via Anaconda simply doesn't work. Neither searching in the search bar of the system. - A: Do you find Anaconda Navigator if you search for it? Yes, okay than it might takes some more time. Especially the first launch takes some seconds up to minutes. Do you see a green circle Yes, I see the green circle - takes longer than i think,setup does not finish - I think it's working but loading looking at the end of your message. Did it work in the meantime? maybe you have to stop the task and try to restart if it takes too long OK I'll try - If it doesn't work you can use an online version for the meantime: https://jupyter.org/try-jupyter/lab/ But it doesn't have all the options like Anaconda. But the first part should be fine Thanks! I'll try it for the start. Please tell me what option should I pick at opening the link? Notebook -> python or sth else? Yes excactly :+1: Thank you! You're welcome! - Can I save this online version of the notebook? - Yes, you can do that, there could be multiple ways to do that, just save it with ctrl+S or download it locally, check any of this works for you. You can click on `File` on the top left menu and click on `Save Notebook` - - ::: ::: success ##### ❓ Are you inside an empty Jupyter note+book? Add a '+' below - Yes: +++++++++++++++++++ - Need time:++ - Need help: - i used lite online version , how to insert data - ##### ❓ Do you need help? Please describe your issue - looks like this for me. Is this OK? - ![](https://biont.biobyte.de/uploads/ae18a424-8caa-4e67-9bd8-ae9a2b47cfa8.png) - You should see the page like instructor. did you run 'jupyter lab' ( without quotes) on terminal - - ok, it works >> Great - - When I click alt and enter opens a new. Now it is ok - But i dont see the fist line executed - it stays s - A: Try Shift + Enter, does that work? -OK -what is the difference between shift + enter or alt + enter? I succeded to calculate but I am curious about the difference. Ah ok. - -The title is in italic. is that - On Silvias screen the title is not italic. If you use `# *First day of Python*`with the asterix, the text will be italic. If you use two `**`before and after, it will be bold what is the sh - how to execute the code? - A: Shift + Enter or you can click on the "Play" icon in JupyterLab or Alt + Enter -perfect, thx - -if anaconda not installed,can jupiter be used? - A: You can install Jupyter without Anaconda, but Anaconda will bring you not only the environment but python packages you might need - i have python but -Print is not green as on Siliva page. A: You might have another version of JupyterLab. If the code works for you either way, than it's fine - Is space important in writting code? No, it is not mandatory, but its a good coding practice to follow some rules for clear interpretation of the code. - The document is on autosave mode? Or should I click somewhere save? - You can save it with some name so that you can open this after that as well. othewise you can lose it once your Jupyter session will end or terminate. - ::: ### First steps with Python #### Variables and Assignment This part corresponds to the contents of the [Variables and Assignment](http://swcarpentry.github.io/python-novice-gapminder/02-variables.html) chapter of the tutorial. :::success ##### ❓ Do you need more time? - yes - no ++++++++++++++++++++++++ ##### Questions: - Space is important before and after =? - no but spaces help you to read - it will work exactly the same, however it is good practice or recommaneded to put space before and after to make you code more clear (you can test the code with and without space as well, just to be make sure :) ok - both age and name are variables right? - Yes, you can use any other name as well for variable, it just to let you know about the work. you can name a variable what you like or want. - I get this error when redefining variables: ``` TypeError Traceback (most recent call last) Cell In[12], line 1 ----> 1 age = age + 1 TypeError: can only concatenate str (not "int") to str ``` - A: You are getting this error bcz you are trying to concantenate integer with string, check you have integer value in age variable like : age = 25 not like age = '25' - - what's the difference between using print(age) or just age? we get the same result - if you use print(age), you will get the output in environment and just age shows the value only in Jupyter cells - -if you execute two time the function age+3, how can you "remove" the extra execution? - it is important to write " print (age)" after a formula - - please we need those notes for using tab - Do you mean how to use tab completion? Tab is very heplful, when you start typeing e.g. a function, you can start with the first letters and press tab. You will get a list of suggestins and you can select e.g. the fuction you where searching for. - - is it important to write #### in the bigining of markdown - A:These are merely some markdown conventions to format your document. Or you can say style guide to define headers, paragraphs and so on. - I still not sure if the document is saved. renamed it with my name, but should I do something else? - A: Just go to File option on top left corner and click on save Notebook or press Ctrl+S yes, perfect, thanks - ::: ::: warning ##### ✏️ Hands-on: Seeing Data Types ##### ❓ What are the data types of the following variables? ``` planet = 'Earth' apples = 5 distance = 10.5 ``` I did something ... sorry. i tried to copy, and I did not know how - string, integer, float - string, integer, floatong number - strings, integer, floating - string, integer, floating number - str, int, float - string, interger, floating number - strin, integer, float - string, integer, floating - string, integer, floating number - str, int, float - str, int, float - str, int, float - str, int, floatstr, - str, int, float. - str,int,float - str, int, float - str, int, float - str, int, float - str, int, float - str, int, float - str, integer, float ##### ❓ Are you done with the practice boxes? Add a '+' below - Yes: ++++++++++++++++++++++++++++++++++++ - Need time:+ - **Need help**: - could you give all codes run to now,together? - can you get all the types in one go? - A: yes you can do that in one go! - A: You can type `%whos` in a cell and execute the cell to get an overview of all your variables with the type included. The `%` indicates that it's a jupyterlab command and not a python command - can you define as many variables in one chunk? - A: Yes absolutely! - I did not get the exercise with planet ... apples - A: The exercise was to check what are the datatypes of the defined variables. For eg. the planet variable is string(str), apple variable is an integer (int) and distance is a float datatype. - Ah, ok, so it was not a command. Indeed, it was a command to evaluate the datatype of a variable. Does this clear your question? - yes. thank you - - what is the short way to run a cell? - Shift+enter or alt+enter - How to copy? with ctrl+c don't work! - A:click on the left side of the Cell and Press Ctrl + C and press Ctrl + V in a new cell -now it woks, sorry - okay great :+1: - ::: ::: warning ##### ✏️ Hands-on: Fill the table showing the values of the variables in this program after each statement is executed. + | Command | Value of x | Value of y | Value of swap | | --- | --- | --- | --- | x = 1.0 | | | | y = 3.0 | | | | swap = x | | | | x = y | | | | y = swap | | | | - - 0-3 is 4 characters: 0,1,2,3, "sodi" ?? - A: When you use string slicing, python indexes elements [start to stop-1], so it will return "sod" which is element 0,1,2 - So the 2nd number is the length of the slice and not the ending position? - A: Python follows this convention for slicing --> slice(start, stop, step) start: Starting index where the slicing of object starts. stop: Ending index where the slicing of object stops. step: It is an optional argument that determines the increment between each index for slicing. Does this help? - Not yet, sorry. What is the index of the letter d in sodium? - The python indexing starts with 0, so the index of letter d is 2 - Executed this: print(atom_name[0:2]) - Result: so (not sod) ?? - This is corrent so as I mentioned python slicing works [start to stop-1], where your start position was 0 (element 's') and stop position - 1 was 1(element "o"), so it returned "so" - AAh, I get it now. What is the logic of this "-1"? - Lets take an example, - first_string = "Hello, World!" - print(first_string[0:5]) - So this will return characters from index 0 up to, but not including, index 5 where Output: Hello - This is how python indexing works by default. :) OK, thx - You can follow Silvia now, she is explaining again! - can you explain again what should we do on this exercise? - and swap should be 1? ok, great - - i print x and get 3? - if you say x = y and the value of y will go to x variable and give you 3 as output for x - ##### ❓ Are you done with the practice boxes? Add a '+' below + - Yes: ++++++++++++++++++++++++++ - Need time:++ - Need help: ::: :::success #### Is it ok to do 10 minutes break: yes: ++++++++++++++++++++++ No: ::: #### Let's come back at 10:45 (CET) :::success ##### ❓ Are you back? - Yes++++++++++++++++++++++++ - No ##### ❓ Any questions regarding what we did until now? - Q1:can you show us the notebook with the exercise? - A: Silvia did I think. It you still need something let us know. - Q2:wher on the keyboard I can find suare brackets? - A: AltGR +F and G [] - A: for me its AltGR + 8 and 9 - A: It differs based on the keyboard configuration you are using. - Q3 :Sorry, I'm a bit confused about Markdown; do we need to have a separate Markdown document open, in addition to the notebook (or can we otherwise activate this?) I.e, When you say 'let's go to Markdown'..found it, thanks! - A: you can continue with same document. it's just the option she click on top panel and select markdown, code etc. Just follow her instruction, and you are good - Q: I have defined: age=20.5. And now: NameError: name 'age' is not defined What could have happened? - A: make sure you execuated age variable previously - sure, made several excercises with it - Just check it again or define it again would resolve your isssue. YES, this helped >>> Great :+1: - Q: After line 99 my line numbering has restarted. Maybe there is a connection? - A: Could you perhaps elaborate the above? - ![](https://biont.biobyte.de/uploads/7a6207bb-fda8-43dd-8c14-da9ec24fb017.png) - A: Not sure why this is happens, but this should not interfere with your coding functionalities. - OK no problem. It works. - Q: - A: - Q: - A: - Q: - A: ##### ❓ Is the speed fine? - Yes:++++++++++++++++++++++++++++++++ - Too slow: - Too fast: + ::: #### Data Types and Type Conversion :::success ##### ❓ Is the speed fine? - Yes+float ing - Too slow: - Too fast: x squared exponent ::: ::: warning ##### ✏️ Hands-on: Fractions ##### ❓ What type of value is 3.4? How can you find out? - float; print(type(3.4))+++ - float number print(type(3.4)) - float - fflot number - floating nmber - float - print(type(3.4)) - print (type(3.5)) - float, print(type(3.4)) - floating number - print(type(3.4)) - float - float, print (type(3.4)) - folat number print(type(3.4)) - float number print(type(3.4)) - float, print(type(3.4)) ::: ::: warning ##### ✏️ Hands-on: Automatic Type Conversion ##### ❓ What type of value is 3.25 + 4? - float - print(type(3.25 + 4)), float - float - float - class<'float'> - float - + - float - float - float , print(type(3.25+4)) - print(type(3.24 + 4))+ - class float - float - float -float - - - ##### ❓ Are you done with the practice boxes? Add a '+' below - Yes: ++++++++++++++ - Need time: - Need help: - always appear this error : TypeError: 'int' object is not callable - I also get an error: TypeError Traceback (most recent call last) Cell In[205], line 1 ----> 1 print(type(3.24 + 4)) TypeError: 'str' object is not callable ``` a = 3.25 + 4 print(a, type(a)) ``` Try this one and let me know it runs fine Sorry, same error: TypeError Traceback (most recent call last) Cell In[209], line 1 ----> 1 print(a, type(a)) TypeError: 'str' object is not callable We are not working with string here so not sure why you are gettign this error. try to check the code above hopefully there is something goinfg wrong there. I also have the 'int' object is not callable eaven doing what you said -integers are automatically converted to floats as necessary in python. oR you can also define them as float like ``` a = float(3.25) + float(4) print(a, type(a)) ``` ::: ::: warning ##### ✏️ Hands-on: Choose a Type ##### ❓ What type of value (integer, floating point number, or character string) would you use to represent each of the following? Try to come up with more than one good answer for each problem. For example, in # 1, when would counting days with a floating point variable make more sense than using an integer? integr 1) Number of days since the start of the year. - integer ++ - integer - int - integer - class int - integer (if hours are needed then floating) - int - integer - string - integer - integer - int - f - int - integer - class int for days ::: ::: warning ##### ✏️ Hands-on: Choose a Type ##### ❓ What type of value (integer, floating point number, or character string) would you use to represent each of the following? Try to come up with more than one good answer for each problem. For example, in # 1, when would counting days with a floating point variable make more sense than using an integer? 2) Time elapsed from the start of the year until now in days. - float - float - float - float - float - float - float float, to count half days? - float - float - float - float - float - float number - float - float - float - floating number -+ - - ::: #### Built-in Functions and Help :::success ##### ❓Do you have questions? - Q: I stil get problems with 'type': Cell In[220], line 1 ----> 1 type(full_name) TypeError: 'str' object is not callable. Is there anything I can do here? - A: Can you try re-defining your variable `full_name` and try again? - I tried again, and I can print it OK but 'type' doesn't work - A: How about print(type(full_name))? - Unfortunately same problem: ----> 1 print(type(full_name)) - Just to confirm, the variables are not mixed up, could you open a new Jupyter notebook and define full_name and try again? - It worked in a new notebook. - A: The issue could be then variable overwriting in your previous notebook. Glad this worked :) You can continue in the new notebook. - Thanks - Q. why this: print(min("a", "A")) gives A as a result? - A: This is interesting, I wondered this could be a question. So, this happens because Python compares characters based on their ASCII values. The ASCII value of uppercase "A" is 65, and the value of lowercase "a" is 97. Since 65 is less than 97, "A" is considered smaller than "a", and thus the min() function returns "A". Interesting right? - Q: We will have these questions and answers at the end of the course? I think there are very helpful - A: We can share siliva's code and exercise you can find here : https://swcarpentry.github.io/python-novice-gapminder/03-types-conversion.html - Thank you - Q:why do we use print if min (1,2,3) also prints 1? - A: difference is when you use print it will show you in output in console and without print option that will displayed as the output of that cell. Using print option is a good practice if you are not working on Jupyter. - Q: why did it work with print (min ('a','A','0'))? - A: The min() function compares the characters based on their ASCII values and returns the character with the smallest ASCII value. ASCII value of a, A, 0 is 97, 65, 0 respectively, so the expected answer is 0 as this is minimum. - Q:is there a shortcut for print or an alias e.g. to type only p - A: No, unfortunately there is not shortcut for printing in Jupyter notebook. - Q:Thank you. It is very interesting this course - A: Glad you are enjoying and learning alongside :) - Q:are there alternatives to jupyter? - A: Do you mean as IDE? if yes there are certain other alternatives to Jupyter such as PyCharm, Spyder, VS Code, Google Collab etc, but for beginners Jupyter is quite user-friendly and interactive. - Q: is there a difference between "" and `´ - A: No not really, when you are defining them for strings. Double and single quotes are quite interchangeable in context of Python. But we need to be careful, when we use them in a sentence as a string. For Eg. if we want to define a sentence He's a quick learner! as a string, we should use `statement = "He's a quick learner!"` - Q:how can I see which libraries I already imported - A: You can use print(sys.modules.keys()), this will give you all imported libraries, Just make sure you added 'import sys' (without quote) before running the above line - Q:Are libraries imported to the notebook or to the single file? - A: They are imported to your current session of the notebook. If you start a new notebook, you will have to re-import all the libraries. ::: #### Libraries :::success ##### Check out Python libraries: https://docs.python.org/3/library/ #### Your comments - We are having fun. ;) - Feel more motivated when you lead code instead of googleing it by myself :) - First time I understand something about programming, thanks :) - very inspiring course, thx - Thank you to all the organizers and helpers. Very nice intro to python and jupyter! - Thanks very much, it is very helpful and informative - Thanks Silvia :) - Thanks for clear explanations - + ##### ❓Do you have questions? - Q:The data mentioned in the beginning of the course .. - A: For the next session you will need to have the gapminder data downloaded:http://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip - Q: this data should be only downloaded and extracted from the archive? or should I do something else? - A: Yes, you will need to have this data on your system locally, once you download this zip file, you can unzip the contents and keep the files ready for the next sesssion! - Q:is there a reason to not import all libraries at the start of a session? - A: This has several reasons, one reason could be overloading of libraries in your current session. Other reason could be cross-compatibility of libraries with each other. So its always recommended to import libraries as you need! - You are really nice, thank you:D - Q:can i unimport a library during a session - A: In python, You can not 'unimport' a library here but you can delete the particular module from your sys library. Ideally this is not advisable, because this could lead to compatability issues incase some library is dependendent on the current one. - Q: - A: ::: Let’s come back at 13:00 (CET) :::success ##### ❓ Are you back? - Yes ++++++++++++++++++++++++ - No ##### ❓ Any questions regarding what we did until now? - Q: - A: - Q: - A: - Q: - A - Q: - A ##### ❓ Is the speed fine? - Yes:++++++++++++++ - Too slow: - Too fast: ::: ### Schedule | Starting time | Duration | Content | | ------------- | -------- | ---------------- | | 12:00 | 1 h | Lunch break | | 13:00 | 25 min |[Reading Tabular Data into DataFrames](http://swcarpentry.github.io/python-novice-gapminder/07-reading-tabular.html) | | 13:25 | 35 min | [Pandas DataFrames](http://swcarpentry.github.io/python-novice-gapminder/08-data-frames.html)| | 14:00 | 10 min | Break | | 14:00 | 35 min | [Plotting](http://swcarpentry.github.io/python-novice-gapminder/09-plotting.html)| | 14:45 | 15 min | Summary + Feedback | :::success ##### ❓ Before we start: Is the screen clearly visible (add '+') - Yes+++++++++++++++++ - Please zoom in ::: :::warning ##### ❓ Do you have any questions about this mornings content? Q:I didn't get the tab command to be faster in typing the code A: ~~When you start writing a command and hit tab it should give you options or directly autocomplete~~. No, it explains the function of the command, maybe is the python version? Sorry is this a question or an answer? Q:When I start writing a command and hit tab it should give you options or directly autocomplete. But now for me it explains the function of the command. Does it depend on the python version? A: Try to re-open the new notebook and check it is occuring there also, it could also be happend due to the conflict behaviour of keybroad for tab. Q:Can you show us again te command for importing data? A: t Q: I installed pd with jupyter via pip install pandas but the same command failed in terminal. Should not the terminal work in the same way? I used pip for all installations for the course. I have pd so it doesnt really matter. I was just wondering why it failed in the terminal. Thank you! A: To make it run on terminal, you have to first open the PYTHON shell or interpreter for python by typing 'python' (without quote) and run import function. Direct import function on terminal does not recognise your command. ::: ### Reading Tabular Data into DataFrames :::success ##### ❓ Jumping in: Were you able to import pandas and load the data? + - Yes (add `+`)+++++)++++++++++++++++++++ - Need more time + - No +++ - Data: :http://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip Q: - What do you mean by loading data? Finding it? Or what? A: - Yes, if you installed the data you should be able to see it in the location you downloaded A: - OK I have it (perfect!) - No (please copy-paste any error message you received) ``` ModuleNotFoundError Traceback (most recent call last) Cell In[3], line 1 ----> 1 import pandas as pd ModuleNotFoundError: No module named 'pandas' ``` - A: if you get this error try doing ```!pip install pandas```, write this into a cell before you import pandas and it should install it. - Did it work? - For me it does not work. I get "/usr/bin/sh: 1: pip: not found" - What operating System do you use ? - Windows, but I am using jupyterlab from posit workbench - Ok it seems you are missing the required libraries and pip too. It might be an issue with the environment paths. I will have to check how to manually install it in Windows. - I solved it! thank you - Great :) - Can you show us again te command for importing data? in the notebook. I have already downloaded. - It is not really importing you need to see the location where the data are located and where you downloaded - I found the path to my folder and I missed the command line to import the data - This is currently explained by Teresa. You will read the data directly from the path where you have stored it. - - csv comma separated values - - my tab is not working now. - It does not work if you are in the wrong directory - Q: I have trouble importing pandas. pip tells me it is installed though A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.0 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. it seems like compatibility issue of NumPy installed on your system and the compiled version of the pandas library. try to run : ``` pip uninstall pandas numpy pip install pandas numpy ``` no error message is good right? :P YES!No news is a good news So Now are you able to import pandas, successfully ? Yes thank you :) couldnt have done this alone >>> All good, happy that it worked for you :) - Q: Why after writing the command (print) the table looks differently? - A: This is because print gives you the real representation as python gives it to you, while jupyter-notebook has a nice way to parse the data into a table. So for python with print() it gives you a string. Also if you would run it without print in a python script it would not give you any output. It really depends on your coding environment. - Thanks for the detailed answer! - Q:I do not know how to find the path - Please follow what Teresa is saying. She is explaining how to get the location - Q:I need to change drive. Working folder is on C:, data is on D:. I see Teresa, but using her method I cannot change drive. Data is loaded though. See pic below. - In this case you might need give an absolute path so you start with D:\path\to\my\data and go all the way to the data. In windows you can open the location where your data is in the file explorer and click on the path and you can just copy&paste the full path. ![](https://biont.biobyte.de/uploads/1d6cfae3-8070-4e6e-a170-49bec8ec2eeb.png) Tried absolute path. - yes it is loaded, so it's OK? - It looks ok, the error is based on a control sequence that it does not know, but with the table it looks like it still works. Sometimes special characters need a \ in front. OK - Q: hve this error: --------------------------------------------------------------------------- ``` AttributeError Traceback (most recent call last) Cell In[17], line 1 ----> 1 pd.read.csv('../Desktop/data') --> Here you have the mistake read_csv AttributeError: module 'pandas' has no attribute 'read' ``` - A: You have a mistake in the code. There is a . instead of _ This is the correct way to use the module: pd.read_csv() - Did it work now? - Q: Sorry, I don't understand the initial '..' part. Can you explain how to start from the C-drive? I.e., I know where my data is, but don't understand the path. - There are two types of paths **absolute** and **relative** paths. If you do `..` that means go back one folder from my current location (so relative to your location, in this case to location from where your jupyter notebook was opened). What you want if you start with C: is the absolute path. So starting from the lowest point of your harddisk to the location of your data. In windows, you can just open the location of your data in the file browser. On the top bar it should show you the full path seperated with > symbols. if you click on it should give you the full path C:/path/to/data, - Can you see this ? I see a link to data separated by /symbols - I've put my data in the C:\Databases\PythonData for a shorter path - and if you do ```pd.read_csv("C:\Databases\PythonData\gapminder_gdp_oceania.csv")``` does that work ? I have to look up the filename ^^ Now like this. - Does this work for you ? - No ([Errno 2] No such file or directory: 'C:\\Databases\\PythonData\\gapminder_gdp_oceania') - did you put the ".csv" at the end? Also just to be sure again. You are using Windows right? And is your data directly in PythonData? otherwise you need to do ```pd.read_csv("C:\Databases\PythonData\data\gapminder_gdp_oceania.csv")```, or whatever folder it is in. ![Uploading file..._15wvebmga]() Ihave this error:--------------------------------- ``` FileNotFoundError Traceback (most recent call last) Cell In[21], line 1 ----> 1 data_oceania_country = pd.read_csv('../Desktop/data/gapminder_gpd_oceania.csv', index_col='country') File ~\anaconda3\Lib\site-packages\pandas\io\parsers\readers.py:1026, in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, date_format, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options, dtype_backend) 1013 kwds_defaults = _refine_defaults_read( 1014 dialect, 1015 delimiter, (...) 1022 dtype_backend=dtype_backend, 1023 ) 1024 kwds.update(kwds_defaults) -> 1026 return _read(filepath_or_buffer, kwds) File ~\anaconda3\Lib\site-packages\pandas\io\parsers\readers.py:620, in _read(filepath_or_buffer, kwds) 617 _validate_names(kwds.get("names", None)) 619 # Create the parser. --> 620 parser = TextFileReader(filepath_or_buffer, **kwds) 622 if chunksize or iterator: 623 return parser File ~\anaconda3\Lib\site-packages\pandas\io\parsers\readers.py:1620, in TextFileReader.__init__(self, f, engine, **kwds) 1617 self.options["has_index_names"] = kwds["has_index_names"] 1619 self.handles: IOHandles | None = None -> 1620 self._engine = self._make_engine(f, self.engine) File ~\anaconda3\Lib\site-packages\pandas\io\parsers\readers.py:1880, in TextFileReader._make_engine(self, f, engine) 1878 if "b" not in mode: 1879 mode += "b" -> 1880 self.handles = get_handle( 1881 f, 1882 mode, 1883 encoding=self.options.get("encoding", None), 1884 compression=self.options.get("compression", None), 1885 memory_map=self.options.get("memory_map", False), 1886 is_text=is_text, 1887 errors=self.options.get("encoding_errors", "strict"), 1888 storage_options=self.options.get("storage_options", None), 1889 ) 1890 assert self.handles is not None 1891 f = self.handles.handle File ~\anaconda3\Lib\site-packages\pandas\io\common.py:873, in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options) 868 elif isinstance(handle, str): 869 # Check whether the filename is to be opened in binary mode. 870 # Binary mode does not support 'encoding' and 'newline'. 871 if ioargs.encoding and "b" not in ioargs.mode: 872 # Encoding --> 873 handle = open( 874 handle, 875 ioargs.mode, 876 encoding=ioargs.encoding, 877 errors=errors, 878 newline="", 879 ) 880 else: 881 # Binary mode 882 handle = open(handle, ioargs.mode) FileNotFoundError: [Errno 2] No such file or directory: '../Desktop/data/gapminder_gpd_oceania.csv' ``` - can I just pich in? It helped my case when i changed all back slashes (\) to forward slashes (/) - if you are using different operating system, you can try like this. Your error says file not found so that means you are not assign the file path correctly. - In windows you usually have to do (\), in linux and macos it is usually (/) but there are windows versions that allow both. - on windows it is most likely ```C:\Users\<Your_Username>\Desktop\data\gapminder_gpd_oceania.csv```, of course you need to replace the ```<Your_Username>``` with your actual one and make sure that C: is your default drive (which is nearly always the case). - You can try to first copy the directory of your data in your drive where you want and then click on File option on jupyter notebook and open the path and create the new notebook in the same path and just provide the file name should work (your script and data should be in same directory in that case) - Can we get a copy of notebook from Teresa? - do you want it now? yes, since i can't keep up . thanks - T for transpose doesnt change the saved file? it is just how it is shown on my screen? - Yes, this is just data manipulation function it does not changes the actual file. here we are just performing some functions on the data.n I think we need an another copy of notebook with the next commands ... still struggle to resolve the issue. I will try to redo after course all comands - You can find all the commands here : https://swcarpentry.github.io/python-novice-gapminder/07-reading-tabular.html -ok, thanks - Very useful advice. Finnaly I can follow the commands from this website after solving the problem with the location. Thanks Silvia ### Pandas DataFrames :::success ##### ✏️ Pandas DataFrames Type along with the instructor to learn about programming style. ##### ❓ Do you need help? Please describe your issue or question - I still cannot load data, it would be extremely helpful to see where to load the full path from a given drive (I have Windows). So I have data, and know whwre, but don't understand how to load it (I'm used to loading in R, starting with full path from . "C-drive"). Sorry, I don't understand the shortcuts :) - Similar question has been answered above, I would recommend you to see line#1171 and try to follow the path as suggested there and let us know again if you encounter any further issue. - Thank you and I tried the above but still not working, will try to just follow the presentation now and read the info on commands later.. Hopefully the final workbook (now being made) can be shared later, with all info/commands, and I'll try to repeat - It seems to work now, I used: TestData = pd.read_csv(r"C:/Databases/PythonData/data/gapminder_gdp_oceania.csv") - ::: :::success ##### ❓ Is the difference between `iloc` and `loc` clear? (add `+`) Yes ++++++++++++++ - No (can you elaborate?)++++ - What was iloc for? Already forgot, sorry. - with the iloc, you access the data using indexes, so integer values. - And with loc you access it using the heading names, right? - yes correct. OK - yes, is clear but could you give an example of combining for instance transpose and a sub-table we made. ++Thanks for showing! ::: :::success Do you need a break? -yes +++++++++++++++++++ -no ++++++++++++++++ ::: #### Let's come back at 14:30 (CET) :::success ##### ❓ Is the `iloc` and `loc` based computations clear? (add `+`) - Yes+++++++++++++ - No + :::success Do you need a break?+ - yes+++++ - no can you leave the screen shared on your notebook page during the break? need to fix last row of code + Need 10 minutes break :) I would like a 10 min break if possible - Q: row is not defined in my case?--> - Can you specify what command you were using when you got this error? - the last command as data.iloc[row,col] data.loc[row,col] col is defined as we define earlier as index_col='country' so for row we have to define as index_row='country name'? - A: when you use loc or iloc you have to specify values. So you need to so for example data.iloc[0,3] or data.loc["Albania", "gdpPercap_1952"], row and col are here meant as placeholders for the specific row and col you want to access. - ok I got it ###### ❓other questions and concerns - can we have a copy of notebook again? I cannot run commands if there are gaps in my notebook. It would be very hlpful if Teresa could give us again her notebook. In this way we could keep up with the next commands. - You can find all the commands here : https://swcarpentry.github.io/python-novice-gapminder/07-reading-tabular.html - is there a list of all possible commands from a library? How can I know in advance which library is better to do a certain work with my data? - can we get the recordings of last session ? - Yes, you all will get a link for all the sessions via email after the workshop. ``` When I write the command: print(data_oceania_country.T) I get this error:--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[61], line 1 ----> 1 print(data_oceania_country.T) NameError: name 'data_oceania_country' is not defined ``` - This error suggests that data_oceania_country does not exist when you do your print. So you will have to rerun the cell that contains the definition of data_oceania_country. So the cell where you read it in. The way jupyter notebook works is that it always only knows in the next cell what was run in a previous cell. So if you do changes above you need to rerun that cell. - but the results from defining data_oceania_country appears good as in the link with the command - Did you try putting the definition of data_oceania_country in the same cell as your print. So data_oceania_country = pd.read_csv(your_path).thx - I succeded, thx - is there a way to specify certain rows/columns which are not a range? E.g. data for Albania + France only - I have to see if there is a simpler command but I would do something like: ```df.loc[df['country'].isin(['Albania', 'France'])]```, sorry ```data_europe_country.loc[['Albania', 'France']]``` should actually work. So that would be the way. - how can we save our notebooks for the participation certificate? - you will have to send it to us tomorrow when the workshop is finished. We will give you more details in the last 20 minutes of the workshop (tomorrow) ::: :::success ##### ❓ Are you back? - Yes++++++++++++++++++++++++++ - No ##### ❓ Any questions regarding what we did until now? - Q1: where can we find the slides Teresa mentioned? - A: You mean this : https://swcarpentry.github.io/python-novice-gapminder/08-data-frames.html#select-values-or-nan-using-a-boolean-mask- --- - Q2: - A: ##### ❓ Is the speed fine? - Yes:+++++++++++++++++++++ - Too slow: + - Too fast: + ::: :::success ##### ❓ Is the computation of the wealth score clear? (add `+`) - Yes++++++ - No+++ ----------------------------------------------- - What is NaN for? I m - ean what is abbreviated. Usually it is n.a. - NaN (Not a Number) is commonly use for missing or null value in the dataset. - Is a mask the same as if I would code x > 10000 true print value? Just as a short handy function? - Yes kind of, but you can use these statements as filters for pandas or other libraries. - is there a rule when to use () and when to use []? - Usually `()` are used for function/method calls while `[]` is used for accessing data. So `mean()` for example is a method call. - I am lost, what is axis again? is it specific for sum? - axis parameter is for an operation is going to performed eithor on row or column. - axis=0 refer to row - axis=1 refers to column - here the code I used: print(data_eu.groupby(wealth_score).sum) - I get this for the last excerise: <bound method GroupBy.sum of <pandas.core.groupby.generic.DataFrameGroupBy object at 0x106dde0c0>> - what call did you use to get this? It returns the result but it is a pandas object. - Try to use sum(), with brackets. - that worked, I thought I used the same command as the instructor - So when you do .sum alone it doesreturn this generic text because it just creates a .sum object. If you do () it is the sum function which returns the sum. - can we get the recording oof last session before tomorrow class so we can practice - unfortunately not. We need to get the video recordings from our partners and you will get only after the workshop. Tomorrow is not focused too much in pandas but if you want you can go throught the material - In Linux there are the man pages for a software. Do you have such a page for these Python/Pandas commands? - In jupyter you should be able to use help(your_command) and of course there are man pages and documentations online for all commands. - oh yes, thx what was the last command for today? was it print(data.groupby(wealth_score).sum())?? yes,print (data.groupby (wealth_score).sum ()) ::: :::success ##### ❓Any other questions - what do we need to do to get the certificate? - you will have to send your notebook to us tomorrow when the workshop is finished. We will give you more details in the last 20 minutes of the workshop (tomorrow) - Is the notebook autosaved, thus we could be able to acces it tomorrow? - You can save it by pressing Ctrl+S ( just to be safe) - In which format do we have to save our notebook? - Save as *.ipynb` ::: ### Feedback All was clear, looking forward for tomorrow session :::success ##### ❓ One thing that was good about today - eveything was very clear. The helpers in the background are a great help, they helped along while Teresa and Silvia explained. The responses were fast and helped alot <3 - Morning session was very clear and had a good pace+ - very clear - working in the active Hedgehog doc, thanks very much to the helpers - very informative lecture - coworking in HedgeDoc - really liked the HedgeDoc and how we were supported during the training. overall it was very clear! - liked the communication via HedgeDoc an - d contents of the course in general - the lecture was very clear and the HedgeDoc was very useful - great job answering our questions in real time and also taking timie to explain the lines of code in jupyter - love the HedgeWork - great job by the presenters and helpers - dear helpers,thank you - Very useful course, patient organisers. It would be useful if the teacher's Jupiter notebook were accessible and we could have a look, scroll back. We cannot scroll back in the Zoom video. - HedgeDoc is a bit caotic. Often hard to find where am I writing, where I should write. - - I really liked the session and I am happy that I could follow up and learn new things - ##### ❓ One thing to improve - please give examples of exact paths for loading data files (then we know what to modify). Morning session was very clear and easy to follow - - some parts a bit slow - It would have been very helpful if the path was clear in the beginning. - I felt like the afternoon session was a bit too bumpy. not as organised and clear as the morning one - - please give us the codes in advance would be very help ful since the pace should be high - you can find all code snipets in the provided lesson materials: http://swcarpentry.github.io/python-novice-gapminder/instructor/01-run-quit.html - shame we couldn't get to all the content - The pace was good in the morning, but in the afternoon I got a bit lost at the end...I think the only thing that made me understand what was I doing is the fact that I know some R programing - The last part introduced the most "new" commands but we spent the least time on it - In the morning we had the excercises to think about what we were doing. in the afternoon we were just trying to keep up - Thank you!! - Bye all and thank you to the organizers for the course ##### ❓ Any other comments? - - if we get the recordings of the last session we could practice it agin to get it better before tomorrow class? - You can follow the recordings of last year. You can find them on the [Lumos BioNT space](https://alpha.lhumos.org/spaces/5) - Should we donwload the notebooks or save? - Please save the notebooks. You will need them after the workshop if you would like to get a certificate. - Don't worry the notebooks are saved automatically you will need to download it but we can do together tomorrow - I'm getting an error while saving: "A connection to the Jupyter server could not be established. JupyterLab will continue trying to reconnect. Check your network connection or Jupyter server configuration." (server connection error) Is there a way to get it fixed? OK I checked the directory and it seems that the notebook was saved automatically at correct time, however it cannot connect to the server now. Hoperfully it'll work later. - a short introduction of upcoming workshops (if any)+ - Tomorrow at the end of the workshop we can do that - (and maybe where to find recordings / materials of previous (not necessary only python)) - You can find them on the [Lumos BioNT space](https://alpha.lhumos.org/spaces/5) - what was the last command for today? was it print(data.groupby(wealth_score).sum())???? - yes - some general Python rules, such as the (), []. when to use a .somethingIdotothedatacommand, - yes, it would be nice to get more explanation on the syntax (after R a bit confused of the order) - That's true - what was the last command for today? was it print(data.groupby(wealth_score).sum())???? - yes -see you -bye - ::: # From zero to hero with Python (**Main**) **Date**: 17-10-2024 - 18-10-2024 **Time**: 09:00 - 15:00 (CEST) **Location**: Online ([Zoom](https://epfl.zoom.us/j/64829799035?pwd=DYKW7EWKUzurBebej4hNaJEIW9F2xn.1)) **Code of Conduct**: [The Carpentries code of conduct](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html) :::info ### Code of Conduct Participants are expected to follow those guidelines: * Use welcoming and inclusive language * Be respectful of different viewpoints and experiences * Gracefully accept constructive criticism and give constructive feedback * Focus on what is best for the community * Show courtesy and respect towards other community members ::: ### Schedule for the workshop | Day | Tutorial | Instructor | | --- | --- | --- | | 1-Thursday morning (17th Oct) | [Programming environment setup until Libraries](http://swcarpentry.github.io/python-novice-gapminder/index.html) | Silvia | | 2-Thursday afternoon (17th Oct) | [Reading Tabular to Plotting](http://swcarpentry.github.io/python-novice-gapminder/07-reading-tabular.html) | Teresa | | 3-Friday morning (18th Oct) | [Lists until Variable Scope](http://swcarpentry.github.io/python-novice-gapminder/11-lists.html) | Rabea | | 4-Friday afternoon (18th Oct) | [Programming Style and exercises](http://swcarpentry.github.io/python-novice-gapminder/18-style.html) | Till | [TOC] ### Recommendations :::info To follow the workshop more efficiently, we recommend having a two-screen setup: for example, one to display the instructor’s shared screen and the collaborative pad, and another one for your own screen ::: :::info ##### ✏️ You can find yesterday's pad in here: + Link is leading to a Sign in page only. Should we register to see the pad? ::: ## Day 2 - Friday morning ### Schedule | Starting time | Duration | Content | | ------------- | -------- | --------------------- | | 09:00 | 15 min | Welcome + Summary | | 09:15 | 20 min | Plotting | | 09:35 | 30 min | [Lists](https://swcarpentry.github.io/python-novice-gapminder/11-lists.html) | | 10:05 | 25 min |[For Loops](https://swcarpentry.github.io/python-novice-gapminder/12-for-loops.html) | | 10:30 | 10 min | Break | | 10:40 | 30 min | [Conditionals](http://swcarpentry.github.io/python-novice-gapminder/13-conditionals.html) | | 11:10 | 25 min | [Looping Over Data Set](http://swcarpentry.github.io/python-novice-gapminder/14-looping-data-sets.html) | | 11:35 | 25 min | [Writing Functions](http://swcarpentry.github.io/python-novice-gapminder/16-writing-functions.html) | | 12:00 | 1 h | Lunch break | 13:00 | 20 min | [Variable Scope](http://swcarpentry.github.io/python-novice-gapminder/17-scope.html) | | | 13:20 | 25 min | [Programming Style]() | | 13:45 | 35 min |[Exercises]() | | 14:20 | 30 min | [Wrap up]() | | 14:45 | 15 min | Summary + Feedback | :::warning ##### ❓ Do you have any questions about yesterday's content? Q: A: Q: A: Q: A: Q: A: ::: :::success ##### ❓ Before we start: Is the screen clearly visible (add '+') Yes +++++++++++++++++++ - ![](https://biont.biobyte.de/uploads/d16f298f-6ba8-4a9e-9d91-aeb6cfd790e4.png) cannot create notebook Please zoom in - ::: :::info ##### ✏️ You can find yesterday's pad in here: https://biont.biobyte.de/PGJN59LSSHitIjCPrni_6g?both ::: :::warning ##### ✏️ Hands-on: Start up your or Jupyterlab Start your Jupyterlab instance and bring up the main page in your browser. Create a new Jupyter Notebook and name it `Day2` ##### ❓ Are you inside an empty Jupyter notebook? Add a '+' below - Yes: + +++++++++++++++++++++++++++ - Need time: - Need help: + - ![](https://biont.biobyte.de/uploads/8ea040c4-755b-4dd3-b66e-3562d9454df8.png) yes, win10 A: Please navigate to different location such as Desktop for example and try? You can go there by clicking on the folder "Users" and then there should be your WIndows User Name, you have to click on that ok working now :+1: ##### ❓ Do you need help? Please describe your issue - - - - ::: :::warning ##### ✏️ Hands-on: Loade tabular data You need to follow the current steps to be able to load the [gapminder data](http://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip) into python. ``` 1. Download the gapminder data by clicking the link above 2. Locate the directiory the gap minder data is located in (e.g. Downloads) 3. Unpack the downloaded file 4. (a)Copy/Move the data to your work directory or (b) get the path to the download location 5a. Open a notebook and type: import pandas as pd pd.read_csv('data/gapminder_gdp_oceania.csv') 5b. Find the relative or the absolute path to your data - Find absolute path: right click on the data folder - under proparties you will find the path - make sure that the data folder is included in the path 6b. Open a notebook and type: import pandas as pd pd.read_csv('<absolute_path>/gapminder_gdp_oceania.csv') - make sure that you replaced <absolute_path> with your path ``` ##### ❓ Are you inside an Jupyter notebook and data loaded? Add a '+' below - Yes: ++++++++++++++++++++ - Need time: - Need help: ##### ❓ Do you need help? Please describe your issue - Should we repeat these steps today? with import pandas as pd .. and so on? there are same data, right? only comands should be repeated on the notebook? - A:Yes, as we have started with a new notebook today, we need to again import all the libraries that are needed. - after command import panda as pd i get this error: --------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[2], line 1 ----> 1 import panda as pd ModuleNotFoundError: No module named 'panda' - - A: Please correct the panda to pandas. This will solve your error. -uh, thx - A: Did that help? yes. thanks - Q: I tried to import my data with the path written with this slash \, it didn't work but now it worked with the slash in the other direction /. - A: The path depends on the system you are using, for Linux based system "/" works, whereas for windows "\" works in the path. Thanks! - Q: I get this error after comand:pd.read_csv('C:/Users/Admin/Desktop/data/gapminder_gdp_oceania.csv) :Cell In[6], line 1 pd.read_csv('C:/Users/Admin/Desktop/data/gapminder_gdp_oceania.csv) ^ SyntaxError: unterminated string literal (detected at line 1) - A: Please check the starting inverted comma and ending inverted comma, I see you have a ' missing in the end. - yes. solved it. thank you very much Great! shouldn't be country? not county? - - - - - ::: :::warning ##### ✏️ Hands-on: Reading Other Data ##### ❓ Read the data in gapminder_gdp_americas.csv (which should be in the same directory as gapminder_gdp_oceania.csv) into a variable called data_americas and display its summary statistics. ##### ❓ Are you done? Add a '+' below - Yes: ++++++++++++++++++++++++++ - Need time: - Need help: + - Q: Server connection error - "A connection to the Jupyter server could not be established. JupyterLab will continue trying to reconnect. Check your network connection or Jupyter server configuration." - A: I think this depends on your internet connection. A way around would be to restart your Jupyter notebook and try if this helps? It works for now, thank you for the help :) - Q : for the americas data, should we also use the index_col = 'country'? - A: Please follow Teresa, she is explaining this. - Q: Desktop/Data/gapminder_gdp_oceania.csv NameError: name 'Desktop' is not defined - A: Please check if this is a complete path to your file. There seems to be something missing before Desktop I'm triyng You are propably in your Desktop already, so the path would be data/gapminder_gdp_oceania.csv On Windows you can find the path by opening File Explorer, click on folder with data and then, click on 3 dots ... and click "Copy Path" on winodws it will be something like /c/Users/your_username/Desktop/data/gapminder_gdp_oceania.csv - A: The other solution could, you can go to the file where it is located, and right click on the file and select Properties, you will see a dialog box like this. ![](https://biont.biobyte.de/uploads/73ee946c-50dc-43bb-9403-13927a8c4b99.png) You can then select the location from here and paste it in the Jupyter lab. - Q:I have the file in downloads, but I copied it on desktop, and I workrd from desktop yesterday without any problems, i copied the path with right click on the file from anaconda - all is settled as yesterday in the new notebook - Solved :))) - now I'm behind with exercise - Q:I have this warning Matplotlib is building the font cache; this may take a moment. - A: Warnings are fine, can be ignored. - Thank you c: - Q:data = pd.read_csv"C:\Users\amirz\Desktop\python-novice-gapminder-data\data\gapminder_gdp_americas.csv" get this errror : syntax errror on C:\ why? - A: You are missing starting and ending paranthesis. Try this thnx - Q:getting this error AttributeError Traceback (most recent call last) Cell In[19], line 2 1 plt.plot(time, position) ----> 2 plot.xlable('Time') 3 plt.ylable('Position') AttributeError: module 'matplotlib.pyplot' has no attribute 'xlable' - A: Please check the spelling, it should be xlabel i assume. ok got it - - Q:got this error - Cell In[20], line 1 ----> 1 data.loc['Austraila'].plot() File /lib/python3.12/site-packages/pandas/core/indexing.py:1192, in _LocationIndexer.__getitem__(self, key) 1190 maybe_callable = com.apply_if_callable(key, self.obj) 1191 maybe_callable = self._check_deprecated_callable_usage(key, maybe_callable) -> 1192 return self._getitem_axis(maybe_callable, axis=axis) - A: Please check the spelling of Australia :) thanks :) ::: ### 1 Plotting :::success ##### ❓ Jumping in: Were you able to import matplotlib? (add `+`) - Yes ++++++++++++++++ - No (please copy-paste any error message you received) ##### ❓ Do you need help? Please describe your issue - Q:import successful but when giving the command plt.plot(time,position)--> name 'plt' is not defined - A: Did you use this command `import matplotlib.pyplot as plt`? yes import matplotlib.pyplot as plt, no error message - Q: now it worked, I needed to restart the notebook after installation of matplotlib and didn't send the command again - A: Great! - Q:how is title of the plot picked? For some plots I got <Axes: > or <2d bla bla>. The plots look great tho. - A: The title you see in some plots is a default output from matplotlib when you do not explicitly set a title for your plot. This happens because matplotlib objects (like Axes and Figure) are being displayed without a custom title, and Python outputs their string representation in your environment. ::: ### 2. Lists :::success ##### ✏️ Lists Type along with Rabea to learn about lists ##### ❓ Before we start: Is the screen clearly visible (add '+') + Yes: ++++++++++++++ Please increase font: ##### ❓ Do you need help? Please describe your issue or question - I get this error: - Did you execute the cell as Rabea did? You can trype `pressures = [0.273, 0.275, 0.277, 0.275, 0.276] - solved it. thx - - - ::: :::warning ##### ✏️ Exercise 3 - Slice Bounds What does the following program print? ```python element = 'lithium' print(element[0:20]) print(element[-1:3]) ``` lithium - lithium +++++++++++++ - lithium, empty ++++++++++++++ - lithium and None - lithium, empty - lithium, nothing - lithium, empty - lithium at each command line - lithium, empty tricky to number from 0 :))) - Q: How do you slice to the last position? n:last position - -1 does not work tried it - print(element[0:-1]) - result: lithiu - ![](https://biont.biobyte.de/uploads/6c5fc713-c500-4463-8475-0aa5e12fa46a.png) - print(element[-1]) - If you are using [0:-1] you will get the slice Rabea: Sorry, you can actually use [0:] to get everything. Does that work? Yes, thx. i get the error:Cell In[122], line 1 print(element[0;-1]) ^ SyntaxError: invalid syntax Solved it : it was ";" indtead of ":" A: Great! - Q: in the "del" example, how come we put del + space and not del. Thanks!+ -A: del is a statment like if, for, and not a build in function like print(). You will learn about for-loops soon. As del is a statment it can directly oparte on the provided expression comming after del. You can also type `del(primes[4]) and it will work. However the '()' are not needed - Q: how can I append a value in a specific position? - A: In python, you can append a value at a specific position using insert() method. To do so, you can use list.insert(index, value) where index is the position where you want to insert a value, and value is the element you want to insert. ::: #### Let's come back at 10:36 (CEST) :::success ##### ❓ Are you back? - Yes+++++++++++++++++++ - + - No ##### ❓ Any questions regarding what we did until now? - Q1: - A: --- - Q2: - A: ##### ❓ Is the speed fine? + - Yes:+++++++++++++++++++++ - - Too slow: - Too fast: ::: ### 3. For Loops :::success ##### ✏️ For Loops Type along with Rabea to learn about for loops ##### ❓ Do you need help? Please describe your issue or question - I get this error when putting "for numbers in [2, 3, 5]"": Cell In[95], line 1 for number in [2, 3, 5]: ^ SyntaxError: incomplete input - Cell In[72], line 1 for number in [2,3,5]: ^ SyntaxError: unexpected EOF while parsin - die you execute the cell directly after the line `for ...` - yes - you are missing the body of the forloop. Please press enter without shift to get to the second line and add 'print(number)'. You need to have somthing 'todo' for the for-loop. - now i get it, solved ^^ - perfect - Q:why you need to have two * ? - A: ** is a exponentiation operation in python. For eg. 2 ** 3, means 2^3 = 8. Is this clear for you? - yes, thank you perfect - Q:I did not understand. We have prime in primes? Why is that? I mean, what is prime? To me if I print prime i have a 5. - A: Please follow Rabea, she is explaining this again - Thankyouu - Q: for number in range (0,3): This "number" could be any word I choose? - A: Yes for the number, also called iterater, you can choose what ever 'name' you want. You will often see an 'i' for index used eg. `for i in range(0,3)` - OK ::: :::warning ##### ✏️ Exercise 3 - Practice Accumulating Fill in the blanks in each of the programs below to produce the indicated result. ```python # Total length of the strings in the list: ["red", "green", "blue"] => 12 total = 0 for word in ["red", "green", "blue"]:> [] ____ = ____ + len(word) print(total) ``` - 12 - 12+ - equals 12 - total=total+len(word) - 12 - total =total + len(word)++ - 12 - total = total+len(word) - total & total (4 underscores are a bit misleading :) - total - total - total = 0 for word in ["red", "green", "blue"]: total = len('green')+ len('red')+ len ('blue') print(total) - total = 0 for word in ["red", "green", "blue"]: total = len('red') + len('green') + len('blue') print(total) - Q: Please write the solution ``` total = 0 for word in ["red", "green", "blue"]: total = total + len(word) print(total) ``` - Why do you need the total+? - total = total + len(word) - This is not logical for me? A=A+len???? - Why not A=len?? A: For every time we get the result of total, we need to update it to the previous total value, hence we use total = total + len (word) A=len will only store the current value of your equation and not update it. - I know I have stupid questions but what is the difference between storing and updating? Until now we could always update any value just by writing A=anything_new - I get this error: TypeError Traceback (most recent call last) Cell In[73], line 2 1 for word in ["red", "green", "blue"]: ----> 2 total = total + len(word) 3 print(total) TypeError: can only concatenate tuple (not "int") to tuple - A: Can you give us the command leading to the error? I missed the line "total=0". Still not understanding why is needed. - you need the total = 0 because then the datatype of total is int. Therefore you can perfome the calculation: `total = total + len(word)` - understood - perfect! Q: could you please give a loop within a loop example? - A: Sure, here it is ``` for i in range(1, 4): #first for loop for j in range (1,4): #second for loop print(i,j) ``` Does this help? ::: ### 4. Conditionals ::: success ##### ✏️ If Statements / Conditionals Type along with the instructor to learn about conditionals ##### ❓ Do you need help? Please describe your issue or question - I tried the wrong order and got this error: - File <string>:4 elif m > 9.0: ^ IndentationError: unindent does not match any outer indentation level - A: Please check if you have proper indentation in your if, elif statements. Could you paste your code here to help better? - for m in masses: if m > 3.0: print (m, 'is large') elif m > 9.0: print (m, 'is huge') else: print (m, 'is small') I see, there is some indentation error in the above code. ``` for m in masses: if m > 3.0: print(m, "is large") elif m > 9.0: print(m, "is huge") else: print(m, "is small") ``` - I tried again and now it works, but looks exactly the same A: Earlier you had less spaces before your if, elif and else statements in comparison to the for statement. thank you - Q: if I want a second condition after the initial "if", should I always use elif? or I can use else? - A: I think it totally depends on conditions you are building, if you have just two conditions, if-else is sufficient. In case when you have more than 2 conditions, you can use if-elif-else. Thank you! ::: :::warning ##### ✏️ Exercise 1 - TRACING EXECUTION: What does this program print? ``` pressure = 71.9 if pressure > 50.0: pressure = 25.0 elif pressure <= 50.0: pressure = 0.0 print(pressure) ``` - 25++++++ - 25 - 25 - 25 - 25.0+ - 25.0 - 25.0 - 25 - 25.0 - 25.0 - 25.0 - 25.0 - 25.0 - 25.0 - - equals 25.0 - - 25 - ##### ❓ Do you need help? Please describe your issue or question - Q: if you can explain again why in the exercise with total and len, we use total=0. Ah ok, thank you - A: Reabea is expalining it again. - Q: when using describe like yesterday on the terms today like "prime" or "pressures" i get an error: print(pressures.describe()): AttributeError Traceback (most recent call last) Cell In[161], line 1 -Q: sorry, I still don´t understand why you don´t have to set the "m" variable in 'for m in masses' A: you actually set the variable in this line "for m in masses", if you use print(m) after the loop it will still be there with the last value of your list in it -Q: but how does it know that m is each part of the list we created? - A: It doesn't really know, but will give m an value, by putting the values from the list into it Ok, thank you :) sure :) ``` for m in masses: if m > 9.0: print(m, "is huge") elif m > 3.0: print(m, "is large") else: print(m, "is small") ``` Here Python will give the variable m the first value of this list "masses" (masses = [3.54, 2.07, 9.22, 1.86, 1.71]) so 3.54 and will go trough the conditionals. Then it will put the next one into m which is 2.07 and so on. So if you execute this loop and use print(m) after, m will still have the last value of the list inside from the last run through the loop, which is 1.71 ----> 1 print(pressures.describe()) AttributeError: 'list' object has no attribute 'describe' - describe is a function of the library pandas. You can not use it outsite. - is this the reason describe is not auto-filled with tab? A: yes, if you don't import pandas in your notebook, the notebook doesn't know the describe method - Thank you!!! - - - - ::: :::warning ##### ❓ Do you have a new pad op - Yes:++++++++++++++++++++++++++++ - need help: ::: :::warning ##### ❓ Do you need help? Please describe your issue or question - Q: pls zoom OUT a little - A: Hopefully the line break makes it more readable now for you? OK - I have an error ``` :<>:1: SyntaxWarning: invalid escape sequence '\g' <>:2: SyntaxWarning: invalid escape sequence '\g' <>:1: SyntaxWarning: invalid escape sequence '\g' <>:2: SyntaxWarning: invalid escape sequence '\g' C:\Users\Admin\AppData\Local\Temp\ipykernel_10648\2338228475.py:1: SyntaxWarning: invalid escape sequence '\g' for filename in ["data\gapminder_gdp_africa.csv", C:\Users\Admin\AppData\Local\Temp\ipykernel_10648\2338228475.py:2: SyntaxWarning: invalid escape sequence '\g' "data\gapminder_gpd_asia.csv"]: C:\Users\Admin\AppData\Local\Temp\ipykernel_10648\2338228475.py:1: SyntaxWarning: invalid escape sequence '\g' for filename in ["data\gapminder_gdp_africa.csv", C:\Users\Admin\AppData\Local\Temp\ipykernel_10648\2338228475.py:2: SyntaxWarning: invalid escape sequence '\g' "data\gapminder_gpd_asia.csv"]: --------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) Cell In[8], line 3 1 for filename in ["data\gapminder_gdp_africa.csv", 2 "data\gapminder_gpd_asia.csv"]: ----> 3 data = pd.read_csv (filename, index_col="country") 4 print(filename, data.min()) File ~\anaconda3\Lib\site-packages\pandas\io\parsers\readers.py:1026 ``` - Sorry, import with /data does not work, I am starting from: pd.read_csv(r"C:/Databases/PythonData/data/gapminder_gdp_oceania.csv"). Not sure how to modify - Please check the location of input file. - yes. i'll do sorry, still on this issue, can't import. I've tried: import pandas as pd for filename in ['data/gapminder_gdp_africa.csv', 'data/gapminder_gdp_asia.csv']: data = pd.read_csv(r"C:/Databases/PythonData/data/gapminder_gdp_africa", r"C:/Databases/PythonData/data/gapminder_gdp_africa", index_col='country') print(filename, data.min()) Error: TypeError Traceback (most recent call last) Cell In[31], line 3 1 import pandas as pd 2 for filename in ['data/gapminder_gdp_africa.csv', 'data/gapminder_gdp_asia.csv']: ----> 3 data = pd.read_csv(r"C:/Databases/PythonData/data/gapminder_gdp_africa", r"C:/Databases/PythonData/data/gapminder_gdp_africa", index_col='country') 4 print(filename, data.min()) TypeError: read_csv() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given - Q: Are both data files saved in the environment? - No, I mean if the asia and africa are saved. Yes the data frames. Alright, thank you! Q: I missed the last 10 minutes. Can someone explain what is happening now? - A: Currently, Till has started a new topic for today's session - Writing Python Functions. Not much missed. You can follow now. I managed to get back on track, thank you :)) A: - you can copy the commands below changing the location of the file ``` import pandas as pd for filename in ['data/gapminder_gdp_africa.csv', 'data/gapminder_gdp_asia.csv']: data = pd.read_csv(filename, index_col='country') print(filename, data.min()) ``` - Q:I have this error:--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[1], line 1 ----> 1 import panda as pd ModuleNotFoundError: No module named 'panda' - A: Please check spelling of pandas - yep - Has this been resolved? yes, thanks. ::: :::warning ##### ✏️ Looping Over Data Sets Type along with the instructor to learn about looping over Data Sets. Make sure to import pandas in your Notebook with `import pandas as pd` ##### ❓ Do you need help? Please describe your issue or question - Q: is the * a placeholder/autofiller? - A: * selects all the files that matches with the specified pattern. - Q: my files are in different order; what the order that is shown depends on? Thank you for the answer :) - A: glob itself does not explicitly sort the results; the sorting behavior depends on the file system. - Q: please show the screen for more time - Q:Error:--------------------------------------------------------------------------- KeyError Traceback (most recent call last) File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3805, in Index.get_loc(self, key) 3804 try: -> 3805 return self._engine.get_loc(casted_key) 3806 except KeyError as err: File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc() File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc() File pandas\\_libs\\hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item() File pandas\\_libs\\hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'gpd_Percap_1952' The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) Cell In[15], line 3 1 for filename in glob.glob("data/gapminder_*.csv"): 2 data = pd.read_csv(filename) ----> 3 print(filename,data["gpd_Percap_1952"].min()) File ~\anaconda3\Lib\site-packages\pandas\core\frame.py:4102, in DataFrame.__getitem__(self, key) 4100 if self.columns.nlevels > 1: 4101 return self._getitem_multilevel(key) -> 4102 indexer = self.columns.get_loc(key) 4103 if is_integer(indexer): 4104 indexer = [indexer] File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3812, in Index.get_loc(self, key) 3807 if isinstance(casted_key, slice) or ( 3808 isinstance(casted_key, abc.Iterable) 3809 and any(isinstance(x, slice) for x in casted_key) 3810 ): 3811 raise InvalidIndexError(key) -> 3812 raise KeyError(key) from err 3813 except TypeError: 3814 # If we have a listlike key, _check_indexing_error will raise 3815 # InvalidIndexError. Otherwise we fall through and re-raise 3816 # the TypeError. 3817 self._check_indexing_error(key) KeyError: 'gpd_Percap_1952' - A: Please check, if `gpd_Percap_1952` is the right keyword defined here? Ah, I wrote gpd - instead of gdp ... but still error - Q: pls show the jupiter -> OK - A: - Q: remind me of the meaning of () behind max or min? Can I put a value in there? - A: max and min are functions and therefore you add the () after them. You can provied paramters of the function with the round brackets. Using the help function you can check what paramters the function max and min have. - Q: Why Path? path (no capital p) gives an error. P works fine though. In "from pathlib import Path" - A: I still have an error: --------------------------------------------------------------------------- KeyError Traceback (most recent call last) File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3805, in Index.get_loc(self, key) 3804 try: -> 3805 return self._engine.get_loc(casted_key) 3806 except KeyError as err: File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc() File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc() File pandas\\_libs\\hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item() File pandas\\_libs\\hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'gdp_Percap_1952' The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) Cell In[21], line 3 1 for filename in glob.glob("data/gapminder_*.csv"): 2 data = pd.read_csv(filename) ----> 3 print(filename, data["gdp_Percap_1952"].min()) File ~\anaconda3\Lib\site-packages\pandas\core\frame.py:4102, in DataFrame.__getitem__(self, key) 4100 if self.columns.nlevels > 1: 4101 return self._getitem_multilevel(key) -> 4102 indexer = self.columns.get_loc(key) 4103 if is_integer(indexer): 4104 indexer = [indexer] File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3812, in Index.get_loc(self, key) 3807 if isinstance(casted_key, slice) or ( 3808 isinstance(casted_key, abc.Iterable) 3809 and any(isinstance(x, slice) for x in casted_key) 3810 ): 3811 raise InvalidIndexError(key) -> 3812 raise KeyError(key) from err 3813 except TypeError: 3814 # If we have a listlike key, _check_indexing_error will raise 3815 # InvalidIndexError. Otherwise we fall through and re-raise 3816 # the TypeError. 3817 self._check_indexing_error(key) KeyError: 'gdp_Percap_1952' - could you provide the code you are executing? for filename in glob.glob("data/gapminder_*.csv"): data = pd.read_csv(filename) print(filename, data["gdp_Percap_1952"].min()) A: Please indent your code after for statement. A modified version is below: ``` for filename in glob.glob("data/gapminder_*.csv"): data = pd.read_csv(filename) print(filename, data["gdpPercap_1952"].min()) ``` - A: it is gdpPercap_1952 not gdp_Percap_1952 - Can you tell us if it works now? still no ``` for filename in glob.glob("data/gapminder_*.csv"): data = pd.read_csv(filename) print(filename, data["gdpPercap_1952"].min()) ``` - A: This is working for me, can you copy this code and check? Finally it works, :) thanks - perfect ::: :::warning ##### ❓ Exercise 6 DETERMINING MATCHES: Which of these files is not matched by the expression glob.glob('data/\*as\*.csv')? ``` data/gapminder_gdp_africa.csv data/gapminder_gdp_americas.csv data/gapminder_gdp_asia.csv ``` - data/gapminder_gdp_africa.csv - africa +++ -ata/gapminder_gdp_africa.csv - africa only - africa, asia - Africa only :+1: - africa as americas and asia have as in it - africa, asia +++ - africa, asia data/gapminder_gdp_africa.csv ONLY data/gapminder_gdp_africa.csv!!!!!!!!+ ASia!!!! -africa + - data/gapminder_gdp_africa.csv ok At the exercise 6 DETERMINING MATCHES: exercise: how phython know that:"NameError: name 'gapminder_gdp_africa' is not defined". At wich step americas, and asia where defined? #### Do you have questions? - Q: Is the Path module for handling files? or better said file and path names, right? - A: yes, the path module helps in flexible handling of files and path. This module is under the pathlib library of python. Q: How do i solve this error while loading data? FileNotFoundError Traceback (most recent call last) Cell In[4], line 3 1 for filename in ["data/gapminder_gdp_africa.csv", 2 "data/gapminder_gdp_asia.csv"]: ----> 3 data = pd.read_csv(filename, index_col="country") 4 print(filename, data.min()) - A: Please check if you have specified correct paths for both the CSV files. Q: in the course I haven't managed to load data in the way that you show, starting with "data\..". I use Windows, and after Google search I found an alternative that worked this morning: data_americas = pd.read_csv(r"C:/Databases/PythonData/data/gapminder_gdp_americas.csv", index_col='country') But I did not understand how to modify this to load data in the last exercise. Could you please show how to load the africa and asia files the absolute path starting from the drive (like above example)? A: What example do you mean? The for loop above? if yes you would have to specify the absolute path for each file, if you want to do it in the exact same manner it was shown. ``` for filename in ["C:/Databases/PythonData/data/gapminder_gdp_africa.csv", "C:/Databases/PythonData/data/gapminder_gdp_asia.csv"]: data = pd.read_csv(filename, index_col="country") print(filename, data.min()) ``` Q: Thanks, though above there are two files mentioned at first, but then just "filename" in L3, do I there list complete paths x2 to replace 'filename'? A: If I understand you correctly. The for will loop over both "filename"s so this means that filename will first take the value of the first item in the list, then the next one in the list. Is this what you mean? -Thanks, will try this again later - At the exercise 6 DETERMINING MATCHES: exercise: how phython know the answer that:"NameError: name 'gapminder_gdp_africa' is not defined". At wich step americas, and asia where defined? - A: In this exercises the question was file names match the pattern: `data/*as*.csv`. Files matching this pattern will be printed. If you want to test this on your notebook you need to see if within your workdirectory you have the data folder. If not you would need to set the correct path. ::: :::success Let’s come back at 13:00 (CET) ##### ❓ Are you back? + - Yes+++++++++++++++++++++++++++++ - No ##### ❓ Any questions regarding what we did until now? - Q1: is this correct? for fname in ['data/gapminder_gdp_asia.csv', 'data/gapminder_gdp_asia.csv']: data_as = pd.read_csv(filename, index_col='country') I think I only get one csv when printing but not both csv files when I repeated the exercise from earlier. - A: You called your variable in the for loop `fname`. Inside the for-loop you need to also use this variable, so try to replace `filename` with `fname`. --- - Q2: - A: ##### ❓ Is the speed fine? - Yes:++++++++++++++++++++++++ - Too slow: - Too fast: maybe keep the Jupiter open a bit longer on the command , thx Agree ::: :::warning ##### ✏️ Writing Functions Type along with the instructor to learn about how to write functions. ##### ❓ Do you need help? Please describe your issue or question - Q: Can you include line brake? In the printed result. - A: Answered live, Yes you can include a line break - Q: Did you executed the def function? - A: Answered live. Yes it was executed - Q: For the code: def print_date(year. month, day): joined = str(year) + "/" + str(month) + "/" str(day) print(joined) this error appear: Cell In[72], line 1 def print_date(year. month, day): ^ SyntaxError: invalid syntax - A: You have use a "." instead of a ",". Replacing it should fix your problem. still Error: Cell In[78], line 2 joined = str(year) + "/" + str(month) + "/" str(day) ^ SyntaxError: invalid syntax maybe "\" should be use instead of "/"? - A: No, you are missing a `+` symbol between `"/"` and `str(day)`. In this case "/" or "\" can be used, it is just a delimiter and any string could be used. (also "+" or "-" or ":" etc...). Just make sure that you have the `""` around the character. - Thank you very much. Now it is ok - Great :) - Q: whhat is the purpose of print(joined)? In the date example - A: print() prints out the results, the way we have used it before. It prints the content of the "joined" variable to your screen. - Q: Cell In[25], line 2 joined = str(year) + '/' + str(month) + '/' str(day) ^ SyntaxError: invalid syntax - A plus is missing. . Python is self explanatory sometimes. If you see there is a symbols that says exactly where the error is. It is good to have this error so we can learn from. - ok - Q:why two == intead of one =? - A: Aswered by Till. If you are using the '==' you want to check if the condition holds. The '=' is used if you want to assign right side to the left sight e.g. as in a math eqation - Q:I got None from print(a) - A:That usually means that `a` was not assigned before, can you check whether you ran the cell where you asign `a`? Yes, I run it, but still gives back None - A: can you paste it here, then I can take a look. ``` def average (values): if len(values)== 0: return None return sum(values) / len(values) a = average([1, 3, 4]) print(a) ``` - A: The first `return` statement is always used and exits the function. You have a `return None` at this point the function exits and will never reach the second `return`. Just remove the `return None`. That should fix your issue. - Q: - A: ::: :::warning ##### ❓ Exercise 7 Identifying Syntax Errors 1. Read the code below and try to identify what the errors are without running it. 2. Run the code and read the error message. Is it a SyntaxError or an IndentationError? 3. Fix the error. 4. Repeat steps 2 and 3 until you have fixed all the errors. ``` def another_function print("Syntax errors are annoying.") print("But at least python tells us about them!") print("So they are usually not too hard to fix.") ``` def another_function(): print("Syntax errors are annoying.") print("But at least python tells us about them!") print("So they are usually not too hard to fix.") RESPONSE: no":" after another_function + no parentheses - SyntaxError: expected '(' > set to another_function(): - I got SyntaxError: expected '(' - - defining the function with (): then print command on same space (like 1 tab) now executable - IndentationError: unexpected indent > correct indentation - fixed the error by writing def another_function(): and then the rest of the code like before and it worked (I also fixed indentations) - - - def another_function(): - def another_function(): - - Insert ":", insert "()", delete a "space" (indentation error). And if you get it right you get a "Syntax error..." and it freaks you out. :grin: - -syntax error in the definition, : missing and () missing and indentation error in line 3 - brackets missing including: e.g. (): - - - - The line def another_function is missing the parentheses () after the function name, which will cause a SyntaxError. Functions in Python need parentheses even if there are no parameters. The second print statement has an extra space before it ( print(...)). Python relies on proper indentation, so this will cause an IndentationError. ::: :::warning ##### ❓ Exercise 8 Definition and use ###### What does the following program print? ``` def report(pressure): print('pressure is', pressure) print('calling', report, 22.5) ``` - calling <function report at 0x000002B8F5C88860> 22.5 - - calling <function report at 0x000001BB5206C400> 22.5 - - calling <function report at 0x000001C6BB5E8860> 22.5 - - - calling <function report at 0x16374a020> 22.5 no :))) maybe if ... return calling <function report at 0x00000215BC772980> 22.5 ###### ❓ How could we improve the code? - - - - - - - ::: :::warning ##### ❓ Exercise 9 Order of operations 1. What is wrong with this example? ``` result = print_time(11, 37, 59) def print_time(hour, minute, second): time_string = str(hour) + ':' + str(minute) + ':' + str(second) print(time_string) ``` 2. After fixing the problem above, explain why running this example code ``` result = print_time(11, 37, 59) print('result of call is:', result) ``` gives this output: ``` 11:37:59 result of call is: None it can be done like this: def print_time(hour, minute, second): time_string = str(hour) + ':' + str(minute) + ':' + str(second) print (time_string) result = print_time(11, 37, 59) ``` try to call the print_time function before it is defined OK: ``` def print_time(hour, minute, second): time_string = str(hour) + ':' + str(minute) + ':' + str(second) print(time_string) ``` `result = print_time(11, 37, 59)` ``` Result: Error: Traceback (most recent call last) Cell In[112], line 1 ----> 1 result = print_time(11, 37, 59) 3 def print_time(hour, minute, second): 4 time_string = str(hour) + ':' + str(minute) + ':' + str(second) NameError: name 'print_time' is not defined print_time should be defined first ``` <function print_time at 0x000001BB5206D4E0> why appear this at exercise 9? - A: Because your function does not `return` it does `print`, so when you call that function within a `print` statement, it will not print a string but it will print the `print_time` object that you see here. -**it would help if we could see the notebook of Till** - A: What exactly? Because I see it on zoom? Do you mean that you want to have the actual notebook? -yes, and solved it. thanks ::: :::warning ##### ❓ Exercise 10 ENCAPSULATION Fill in the blanks to create a function that takes a single filename as an argument, loads the data in the file named by the argument, and returns the minimum value in that data. ``` import pandas as pd def min_in_data(____): data = ____ return ____ ``` #### Solutions: ``` filename, pd.read_csv(filename), data.min().min() ``` ``` -Import pandas as pd def min_in_data(filemame): data = pd.read_csv(filemame) return data.min() ``` ``` import pandas as pd def min_in_data(path): data = pd.read_csv(path) return data.min() ``` ``` - import pandas as pd def min_in_data(pd.read_csv(filename)): data = data.min(filename) return min_in_data(data) ``` ``` def min_in_data(filename): data = pd.read_csv(filename) return data.min() - ``` ``` - ``` ::: :::warning ##### ❓ Exercise 11 Find the first negative number in a list Fill in the blanks to create a function that takes a list of numbers as an argument and returns the first negative value in the list. What does your function do if the list is empty? What if the list has no negative numbers? ``` def first_negative(values): for v in ____: if ____: return ____ ``` #### Solutions: ``` def first_negative(values): for v in values: if v < 0: return v ``` ``` def first_negative(values): for v in (values): if v<0: return v ``` ``` def first_negative(values): for v in values: if v < 0: return v ``` ``` def first_negative(values): for v in values: if v < 0: return v elif len(values)==0: print('the list is empty') else: print('there are no negative numbers') ``` ``` def first_negative(values): for v in values: if v < 0: return v ``` ``` def first_negative(values): if len(values) == 0: print('the list is empty') for v in values: if v < 0: return v for v in values: if v > 0: return print('no negative values') ``` - A: You don't need the last for loop, you could just return None then, since the task of the function is to report the first negative and if there are no negative values in the list, simply return None ``` def first_negative(values): if len(values) == 0: print('The list is empty') return None for v in values: if v < 0: return v print('There are no negative numbers') return None ``` - A: This would work yes. ::: :::warning #### Do you have questions? - can you show the notebook for the last part? - A: sure - TypeError: first_negative() takes 1 positional argument but 4 were given print(first_negative(-2,0,4,5)) - A: This happens because you did not enter a list. You need to put `[]` square brackets around your numbers. So `print(first_negative([-2,0,4,5]))`. - OK another error: 1 first_negative[-2,0,4,5] - A: you still need the `()` brackets that are part of your function. See my example above. `()` calls the function. `[]` defines the list. So in python a list is always written with the square brackets, like `["a", "b", "c"]` ``` TypeError: 'function' object is not subscriptable ``` - A: This you get when you try to use `[]` on a function. You need to use `()`. if you want to call a function with a list as a parameter you need to do `function([1,2,3])` both bracket types are needed. - Could you highlight the best solution? there are like 5-6. Which is your example above? - A: It is hard to say what is the "best solution" here. The shortest working one is: ``` def first_negative(values): for v in values: if v < 0: return v ``` My extensive example is: ``` def first_negative(values): if len(values) == 0: print('The list is empty') return None for v in values: if v < 0: return v print('There are no negative numbers') return None ``` - A: If you are refering to how to execute this, I meant: `print(first_negative([-2,0,4,5]))` - can you show us how to correct the issue with the "temperature" not being defined? Thanks - A: Till answered. You can store the temperature first to a variable and then print this variable: ``` pressure = 103.9 # global variable. def adjust(t): # Define local variable temperature (only accessible inside function). temperature = t * 1.43 / pressure return temperature # Outside the function again. temperature = adjust(0.9) print('adjusted temperature:', temperature) ``` - could you post the entire code, please? I still get temperature not defined with the code from above. - So, how to avoid this error?"NameError: name 'temperature' is not defined" should be temperature defined first? or ... - A: Yes you have to define it first, i.e., assign some value to it first. Normally, if you define a variable inside a function, it is not accessible outside this function (i.e. outside this scope). Exception would be if you define a global variable (like `pressure` in the example). - - - ::: ### 8. Programming Style ::: warning ##### ✏️ Programming Style Follow the instructor to learn about Programming Style. ##### ❓ Do you need help? Please describe your issue or question - would you save data/tables in between or just work with them in python and save only the end result? - A: This really depends on your application. It can be useful to have intermediate results for debugging. Oftentimes you only save your input data and your output data. But again, this really depends on what kind of appliction/program you write. - A: Sometimes you also save the intermediate tables, so that you can use a different python script to read and manipulate them further. - how can I save tables as csv? - A: you could for example use pandas. If you have a dataframe `df` you can just do `df.to_csv('output.csv')` - - - ::: :::info ##### ✏️ KEYPOINTS Programming Style - Follow standard Python style in your code like [PEP](https://peps.python.org/pep-0008/) ::: ::: warning ##### ❓ Do you have any open question about the content of the whole workshop? - - Could we go back to Exercise 10 ENCAPSULATION? exactly, see the code and result, mine is a bit strange for me ![](https://biont.biobyte.de/uploads/a9bb4bd5-24bf-4f40-90cc-339ffcb0500a.png) ah ok, thx!!q - A: follow Till - Beginner question: should we practice coding using Jupyter in the future? what should be the next level after this course in order to Python programming? - A: The notebooks are great for learning and testing code snipeds. If you want to start writing bigger programs you should use a IDE. I am using 'visual studio code'. Spyder maybe? Thanks :) It was very instructive this course and I want thank you all for the promptness with which you answered our questions, and how much I learned -if we have different tables where the data we're interested (eg frequency of mutation in a population) in appears in different rows and we would like to create a summary table with all the other tables, can python be useful? ^^ thanks! - A: Yes you can also merge different dataframe tables using Pandas. As Till pointed out, if you know more or less precisely what you need, it makes sense to try ChatGPT and ask it for the Pandas code. This usually works quite well. -Q Does def always end with return? - A: No, aswerd by Till - If we want to interrogate a biological database ... we should download it exactly as we downloaded the "data" for this workshop? - A: If you can download the data from the database in a table format, it will work like this. Pandas has a lot of functions to work with such table files. You just load the data table file into a Pandas Dataframe, and then you can use all the Pandas functions. - Thanks for the nice course, I missed the information on the how we recive the certificate of this workshop. should i register my name some where? or my registration in the course is sufficient? - Unfortunately I have to leave the course early, can you leave instructions for returning the notebook written? Thank you for the very helpfull course! :) - sure have a good day! pdf not works for me. HTML it is not ok? - Q Where should we sen it to? is *.ipynb OK? No, ther is no Python file. ![](https://biont.biobyte.de/uploads/4e602fd7-09a3-43df-a985-2a5cbbdf6744.png) ::: ipynb - same here I can export as a ton of formats but not .py - is .ipynb fine??? :::info ### Pandas Cheat Sheet https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf ::: Write us your email, please Do we have to send here helpdesk@cecam.org ? I dont have option to download as py file neither I -I also have only .ipynb to downlowd, yes its directly download itas ipynb, if i save it also save the same way - I can download it as .ipynb - Me neither. me neither only .ipynb export as executable script? To Which email address should we send the files? , - where do we send - i don't have .py in the download options. can i send the ypynb file? I have html and others but not the py one - i have save notebook as, maybe I can change the extention from there It is fine to send HTML, NOTEPAD, PDF, latex YES, SAVE AS for save as there is just the file name. can we just change the ending? no not py for me this appear after save as: ![]![](https://biont.biobyte.de/uploads/a3dda2ca-e72c-43f9-a14d-cf4796b11332.png) It is ok asas .ipynb" to which emai we are sending the notebooks? :::info Please download as and send `.ipynb` or `.py` format. ::: ![](https://biont.biobyte.de/uploads/1f6819d7-d639-4178-a695-8f6e09d14905.png) -where do we send the notebooks? to which email? thanks it will return error Thanks Teresa! :) problem solved indeed. which email? Thanks!!! Thank you everyone!!! ### Feedback :::success ##### ❓ One thing that was good about today - All instructors were very kind and helpful. I learned a lot, would love another one. From Zero to Superhero! - I am happy that I learnt a bit about loops and functions! I appreciate that the course was more than basic python - Pacing and contents - pace was slower and it was easier to follow. I never programmed in my life and I managed to follow quite well, much better than expected . Thanks! This course was very useful and clear. I have learned a lot. Thanks Very nice course! I learned a lot.Thanks!! - very helpful workshop with direct feedback here in this document, thanks for patient and helpful feedback. ##### ❓ One thing to improve - More excercises - In case you can tell people ahread of time to put exercise data in a specific folder and show the path how to get there (in Windows, linux etc), this would be very helpful and save time. + - - ##### ❓ Any other comments? - Collaboration through hadgedc is realy nice - - Really liked the realtime support via hedgedoc and thanks for the option of having an helpdesk before the workshop, thanks to all instructors and helpers! - ::: :::info #### Learn how to build a website with GitLab If you want to learn more we recomend: 1. [Building Websites with GitLab](https://biont-eu.gitlab.io/workshop-building-websites-with-gitlab/) 2. [Videos of the parst course are accasable on th BioNT Lhumos space](https://alpha.lhumos.org/spaces/5) ::: :::info #### Please fill the post-workshop survey Please fill the [post-workshop survey](https://survey.bio-it.embl.de/249921?lang=en) ::: :::info #### How to get a certificate 1. Complete the post-workshop survey 2. Send your notebooks of this workshop (having your name or unique identifier in the file name) to `digiorgio@zbmed.de` Deadline: Wednesday 23th of October at 14:00 (CET). Please kindly note that this deadline is final and later submissions will **not** be taken into account! ##### ❓ Questions? - Q:Yesterday I rewrote one of my notebooks, is it fine for the certificate? the first one was a mess - Q: Where is the survey? - A: [post-workshop survey](https://survey.bio-it.embl.de/249921?lang=en) - Q: As I couldn't open Jupyter Lab at the beginning yesterday, I used the online version. The output file, however, is .json type and no other format could be selected. Is it OK to provide you this file? (later files are normal notebook output .ipynb) - A: you could try to open this .json file with your notebook and than export it. If you are not successfull please let us know via email. - Can you give us the email where we can apply for BioNT ambassadors? Do we need any formal qualifications? - There are no formal qualifications needed. You can email Silvia `digiorgio@zbmed.de` and she will provied you further informaiton. -How do i find my unique identifier? - A: You created the identifier while filling the pre-workshop survey via the statment below: - where do i find my unique identifier? - A: You created the identifier while filling the pre-workshop survey via this statment: ``` Number of siblings (as numeric) + First two letters of the city you were born in (lowercase) + First three letters of your current street (lowercase). This will help us match your pre- and post-workshop answers in an anonymized way. ``` - Does it have to be the same unique identifier as in the registration? - A: Yes - is the unique identifier the name we used to enter the zoom meeting? - A: The unique identifier was created for the registation while filling the pre-workshop survey - Q: How do we access the Lumos platform with the materials? - A: [BioNT Lhumos space](https://alpha.lhumos.org/spaces/5) -pdf format it is ok? for the notebook that should be sent? - A: please sent the file in .py or .ipynb data format - perfect, thanks - Thank you -when I send the notebbok, do i have to send also the unique identifier? - A: yes please also sent the unique identifier - THANK YOU , BYE -thanks everyone, see you next time! :::