Event Report: April Pune Python Meetup

April Pune Python Meetup (@PythonPune) was conducted on April 30, 2016 at Redhat, Pune. Around 70 people registered for the meetup but the turnout was around 72-73. A few people registered on the spot.

Python Pune Meetups are organised by Chandan Kumar (@ciypro) who is a fellow RedHat employee, a python programmer and FOSS enthusiast who has contributed to many upstream projects.

The meetup started around 10:45 with the introduction where everybody introduced themselves. Almost everybody knew python. There were 1-2 people who did not know python. There were a few people who were experience in machine learning and some who were completely new to Machine Learning. I had a course on machine learning in my college where i learnt the theory and did some practical assignments in R language. The crowd was diverse consisting of students, data scientists, professors and people of various age groups 18 – 70.

This speakers of this meetup were Satish Patil (@DataGeekSatish) and Sudarshan Gadhave (@sudarshan1989) who took a session on Introduction to Machine Learning. 

4
Satish Patil in Pune Python Meetup

 

5
Sudarshan Gadhave in Pune Python Meetup

Satish Patil is the Founder and Chief Data Scientist of Lemoxo Technologies, Pune where he advises companies large and small on their data strategy. He has 10+ years of research experience in the field of drug discovery and development. He told a few real life machine learning examples from his field in the meetup!

Satish is passionate about applying technology, artificial intelligence, design thinking and cognitive science to better understand, predict and improve business functions. He has a great interest in Machine Learning, Artificial Intelligence, Data Visualisation, Big Data.

Satish covered the following topics:

  • What is Machine Learning
  • The Black Box of Machine Learning
  • features
  • training and test data set
  • classification
  • clustering
  • pure and impure states
  • entropy
  • decision tree
  • supervised and unsupervised learning
  • market basket analysis
  • data pre-processing
  • Titanic data set
  • K means algorithm

Although Machine Learning is a vast concept and it definetly requires more sessions to grasp, but Satish made a remarkable effort in making us understand all the above topics in layman terms.

There are a lot of books, courses, material available online for Machine Learning, so why this meetup? Well, the best part about this meetup was the way Satish explained the BUSINESS CONTEXT of MACHINE LEARNINGThis was something new for me to learn. Getting to know the real life examples from the entrepreneur-cum-data scientist was really interesting.

1.jpg
The Machine Learning Workshop in Pune Python Meetup

The details of his talk will be in my next blog.

Chandan Kumar talked about Fedora Labs. The Fedora science spin comes pre-installed with essential tools for scientific and numerical work like IDE, tools and libraries for programming in Python, C, C++, Java and R. It basically eliminates the need to download a bunch a scientific packages which you need.

If you need any help regarding the spin, you can get help from #fedora-science channel on Freenode on IRC.

As Chandan Kumar ALWAYS encourages us to contribute to open source, he introduced us to WHAT CAN I DO FOR FEDORA?. Pune Python meetups and Devsprint are a great platform to seek for help if you want to contribute to opensource.

3
Chandan Kumar in Pune Python Meetup

 

Thanks to Satish Patil and Sudarshan Gadhave for conducting an awesome workshop! We hope to see more such workshops by you in the meetups.

Thanks to RedHat for the food, beverages and venue.

Thanks to Chandan Kumar, Pravin Kumar (@kumar_pravin), Amol Kahat, Sudhir Verma for organising such interesting meetups where we always learn something new 🙂

 

 

Advertisements
Event Report: April Pune Python Meetup

Asynchronous programming

I spent today learning about asynchronous programming and these are my notes related to it.

So, there are two kinds of systems -> synchronous and asynchronous.

In a synchronous system, you wait for a task to finish COMPLETELY before you move on to some other task

In an asynchronous system, you move on to some other task before it finishes. This allows more parallelism.

Event loop

  1. If we want to do something asynchronously in programming language, we use event loop
  2. Event loop can do following things:
    • register tasks to be executed
    • execute the tasks
    • delay execution of tasks
    • cancel tasks
  3. Every event is attached to event listener or else the event gets lost
  4. The main purpose of an event loop is:
    • run first function
    • while that function waits for IO, event loop pauses it and runs another function
    • when the first function returns result then it resumes it

 

Generators

  1. used to create iterators
  2. generators return multiple items but NOT as a list. They return items one by one

 

Difference between a normal callback function and a generator:

Normal callback function approach Generator approach
 After collecting ALL the results, it displays them altogether  It displays result as it finds them.

Advantages:

  • space efficient (no need to store all data at once)
  • time efficient (iteration may finish before all items are needed)
  • user-friendly (allows parallelism)

 

Yield

A function becomes a generator when it uses “yield”

 

Difference between a normal return statement and yield statement:

 Return  Yield
In a function which uses  return statement,

  • local variables are destroyed
  • the scope is destroyed
  • if function is called again, fresh set of variables are created
In a function which uses a yield statement,

  • local variables are not destroyed
  • scope is preserved
  • we can resume the function where we left off

 

Coroutine 

  1. Coroutine is basically used to allow to execute several tasks at once through non-preemptive multitasking
  2.  It passes control to each subroutine, wait for it until it finishes off, we can re-enter the routine later and continue
  3. Coroutine can suspend itself. But once it actually exits/returns, then it cannot be resumed.
  4. There is no need to add a yield statement in a coroutine but the function called by a coroutine can have a yield statement

 

Future

This is the most interesting concept.

  1. Future is one way to write asynchronous code
  2. Future is result of work that has not been completed yet
  3. future() method does not return a result, but returns a future object. When the task completes, the result is returned eventually. Meanwhile, next code is executed.
  4. When do we know that state of future has changed?
    • when set_result() is called
  5. How to check that the task taken by future has been completed?
    • by using event loop -> it watches state of future object to indicate that its done
  6. Future is a way of performing many tasks in parallel, in efficient, non-blocking way
  7. There are two cases:
    • When computation of task does not complete -> future does not complete.
    • When computation of task completes and returns either a result or exception -> future completes
  8. The result of returned by future() can be:
    • value -> future completes successfully
    • exception -> future has failed with an exception
  9. Future has an important property.
    • It can be assigned only once
    • Once it has been given a value, it becomes immutable and can never be over-written

 

Task

  1. a subclass of future
  2. wraps a coroutine
  3. when coroutine is finished, it returns result then task is finished

 

 

Conclusion

  1. For asynchronous programming, we need event loop
  2. We register our tasks/ futures in the event loop
  3. The event loop schedules them and executes them
  4. Callbacks are used so that we are notified when tasks/ future return results
  5. Coroutines are wrapped in tasks/ futures
    • when yield is finding, coroutine is paused
    • when yield gets a value, coroutine continues
  6. If coroutine doesnot return a value but returns an exception, then task fails.

 

This is a simple program i tried with Python’s asyncio module:

Screenshot from 2016-04-27 17-41-52

 

Python asyncio module

 

@asyncio.coroutine:

basically defines a coroutine

 

loop = asyncio.get_event_loop()

creates an event loop

 

loop.run_forever()

runs  a loop until stop() method is called

Asynchronous programming

April Fedora Meetup, Pune

The Fedora meetup was conducted by Kushal Das on 23rd April 2016. Around 15 people attended the meetup at Kushal’s place.

In this meetup, Kushal introduced us to Unit Testing. Its a term often heard in Software development.

 

Testing

Testing is done to determine if there are errors in the code. It does not prove that the code is correct but it just checks that the conditions are handled correctly. Tests are as important as implementation itself.

 

Unit testing

Unit testing is done to evaluate a particular code component. These components can be function , class etc. It determines how well the component reacts to valid or invalid input.  Unit tests are written and executed by software developers.

 

unittest – an automated testing framework in Python

I installed the unittest module on my Fedora machine using:

pip install unittest

I used unittest module in my python program using:

import unittest

 

Basic Test Structure

There are two parts of a basic test:

  1. The code to prepare for test (test fixture)
  2. The code to test

 

Our first Test Case

This program tests cat command in Fedora. The cat command is used to read the contents of a file. We use the cat command in the following way:

Screenshot from 2016-04-25 14-47-30.png

This following program is the first test case. It prints:

  • the output of cat command
  • the error
  • the exit code

 

f2

I saved the script with the name “firetest.py”. The simply ran the script directly from the command line:

Screenshot from 2016-04-25 15-05-45

The above result shows:

  • The output of cat command.
  • The command ran successfully so error message is empty
  • exit code is “0”(successfully completed)

After that, there is a “.” before the dashed line above which indicates that the test passed.

Next, it shows the amount of time the tests took.

After that, the test outcome i.e. OK is shown.

Every test has 3 possible outcomes:

  • OK: the test passes
  • FAILED: the test doesnot pass  and raises “AssertionError” exception
  • ERROR: the test raises error other than “AssertionError”

Now, i did modification to the above program and changed the file name to something else i.e. gave a wrong file name and then ran the script. This is the result:

Screenshot from 2016-04-25 15-18-16

Here, since the file name is wrong, the cat command throws an error. The output is empty. The error message is displayed along with the exit code “1”(unsuccessful completion).

Kushal told us that the “b” before the output and error message indicates that it is in “bytes”.

To convert the output in string, we inserted the following line before print statement:

Screenshot from 2016-04-25 15-23-47

 

ASSERT

Assert statement is used for verifying expectations or some property of code being tested is true. Assert literally means “to state as fact”.

Assert statement stops the routine and sends the error message.

There are many assert statements like:

  • assertEqual
  • assertNotEqual
  • assertTrue
  • assertFalse
  • assertIn
  • assertNotIn
  • assertIs
  • assertIsNot

In the meetup, we tried “assertIn”. assertIn(a, b) checks if a is in b.

This following is the code modification that we made. Here, the assertIn checks if the word “fedora” is present in the output.

Screenshot from 2016-04-25 15-54-57.png

This is the result:

Screenshot from 2016-04-25 15-56-31

This means that the string “Fedora” is present in the output of cat command.

We modified the string “Fedora” to generate an error in this way:

Screenshot from 2016-04-25 15-58-45

After running the script, this is the result:

Screenshot from 2016-04-25 15-59-57.png

The test fails as the string “Fedra” is not present in the output.

Kushal, Chandan Kumar and Praveen Kumar gave all of us a simple task of testing.

My task was to:

  • change a file’s SELinux permissions to root (not through mode)
  • Write a test case to check if it gives an error when i try to access it through normal user sudo chmod 700
  • Check if the error message is present in the SELinux’s log file.

 

In this meetup i learnt about unit testing in Python; also I learnt about SELinux for the first time while solving the task.

Thanks to Kushal for providing the yummy snacks 🙂 and conducting the meetup.

April Fedora Meetup, Pune

Running fedocal instance

Fedocal

A web based calendar application for Fedora.  It is a single place where you can keep track of:

  • meetings
  • events
  • deadlines

 

Get the project

Fork the Fedocal project from Pagure.

Clone the project:

25

Then, go to the Fedocal directory:

Screenshot from 2016-04-18 19-14-45

 

Copy configuration files

12

 

There are 3 things to be done now:

1. Set Secret Key in configuration file

Install pwgen to generate 50 characters long random key:

21

Generate secret key:

22

Put the secret key in fedora.cfg file:

Screenshot from 2016-04-18 21-52-25

 

2. Adjust alembic configuration file:

go to /etc

17

make fedocal directory:

18

create alembic.ini file:

19

copy all data from alembic.ini in your cloned fedocal directory to this /etc/fedocal/alembic.ini

You can directly do this using “cp” command as well.

If you skip this step, then you might get this error while creating database scheme:

16

 

3.Set the database URL

In the alembic.ini file in your CLONED fedocal repo, you will find the line,

sqlalchemy.url = sqlite:////var/tmp/fedocal_dev.sqlite

In the fedocal.cfg file in you CLONED fedocal repo, you will find the line

DB_URL=sqlite:////full/path/to/database.sqlite

Copy the sqlalchemy.url of the alembic.ini file to DB_URL of fedocal.cfg file as given below:

23

If, you skip this step and do not set the database URL, you get this error:

20

 

Install dependencies

Before creating database scheme, I had to install the following dependecies:

3

Even after installing python-openid, I was getting this error:

8.png

Now, according to this article, pycurl fetcher takes precedence if pycurl is installed. So, i checked if pycurl was installed on my machine using the command:

7.png

I removed pycurl using the command:

5

After that, I installed the remaining dependencies:

9101112131415

 

Create database scheme

24

 

Run the server

Screenshot from 2016-04-18 21-37-29.png

 

Now, access the server at http://localhost:5000 in your browser

Screenshot from 2016-04-18 21-40-08.png

Running fedocal instance

Running ircb instance

ircb is a irc bouncer. A bouncer connects to irc server and maintains a persistant connection. Instead of connecting to IRC server, connect to the bouncer. Even if you disconnect your client, it stays connected. If you log in after some time, it shows you the messages that you have missed.

You can find the source code of ircb here.

Following are the screenshots to show the steps i had done to run ircb instance:

Screenshot from 2016-04-17 18-43-31
Clone the repo

 

Screenshot from 2016-04-16 18-10-23
Install dependencies

 

Screenshot from 2016-04-16 18-15-03
Create virtual environment

 

Screenshot from 2016-04-16 18-16-59
Activate virtual environment

 

Screenshot from 2016-04-16 18-17-59
Install requirements

 

Screenshot from 2016-04-16 18-23-09
Create custom location directory

 

Screenshot from 2016-04-16 18-23-57
Create settings file in custom location

 

Screenshot from 2016-04-16 18-24-37
Copy settings

 

Screenshot from 2016-04-16 18-25-13
Export settings

 

Screenshot from 2016-04-16 18-25-39
Run setup.py

 

me1
Create a new user

 

me2
Create network for that user

 

runserver
Run server
1
Open the endpoint using a IRC client.

Here I have used Hexchat which is a popular IRC client for Fedora. You can use xchat, irssi etc

I have used the most popular IRC network Freenode. Freenode listens on port 6667 for non-SSL and 6697 for SSL.

 

2
Add information for your bouncer

Servers of this IRC network: irc.freenode.net 6667

You can also add channels under “autojoin” tab.

3
Screenshot 1: connecting to server

 

4
Screenshot: Connected to server

 

5
Joining channel1 : #dgplug

The last step is to add some channels. Use the command “/join #<channel name>” to join a channel.

<channel name> is the name of a channel you want to join, such as #fedora, #dgplug, #waartaa.

6
Joining channel2: #waartaa

Now you never have to worry about missing another message in an IRC channel again!

Running ircb instance