Asynchronous programming

I spent today learning about asynchronous programming and these are my notes related to it.

So, there are two kinds of systems -> synchronous and asynchronous.

In a synchronous system, you wait for a task to finish COMPLETELY before you move on to some other task

In an asynchronous system, you move on to some other task before it finishes. This allows more parallelism.

Event loop

  1. If we want to do something asynchronously in programming language, we use event loop
  2. Event loop can do following things:
    • register tasks to be executed
    • execute the tasks
    • delay execution of tasks
    • cancel tasks
  3. Every event is attached to event listener or else the event gets lost
  4. The main purpose of an event loop is:
    • run first function
    • while that function waits for IO, event loop pauses it and runs another function
    • when the first function returns result then it resumes it



  1. used to create iterators
  2. generators return multiple items but NOT as a list. They return items one by one


Difference between a normal callback function and a generator:

Normal callback function approach Generator approach
 After collecting ALL the results, it displays them altogether  It displays result as it finds them.


  • space efficient (no need to store all data at once)
  • time efficient (iteration may finish before all items are needed)
  • user-friendly (allows parallelism)



A function becomes a generator when it uses “yield”


Difference between a normal return statement and yield statement:

 Return  Yield
In a function which uses  return statement,

  • local variables are destroyed
  • the scope is destroyed
  • if function is called again, fresh set of variables are created
In a function which uses a yield statement,

  • local variables are not destroyed
  • scope is preserved
  • we can resume the function where we left off



  1. Coroutine is basically used to allow to execute several tasks at once through non-preemptive multitasking
  2.  It passes control to each subroutine, wait for it until it finishes off, we can re-enter the routine later and continue
  3. Coroutine can suspend itself. But once it actually exits/returns, then it cannot be resumed.
  4. There is no need to add a yield statement in a coroutine but the function called by a coroutine can have a yield statement



This is the most interesting concept.

  1. Future is one way to write asynchronous code
  2. Future is result of work that has not been completed yet
  3. future() method does not return a result, but returns a future object. When the task completes, the result is returned eventually. Meanwhile, next code is executed.
  4. When do we know that state of future has changed?
    • when set_result() is called
  5. How to check that the task taken by future has been completed?
    • by using event loop -> it watches state of future object to indicate that its done
  6. Future is a way of performing many tasks in parallel, in efficient, non-blocking way
  7. There are two cases:
    • When computation of task does not complete -> future does not complete.
    • When computation of task completes and returns either a result or exception -> future completes
  8. The result of returned by future() can be:
    • value -> future completes successfully
    • exception -> future has failed with an exception
  9. Future has an important property.
    • It can be assigned only once
    • Once it has been given a value, it becomes immutable and can never be over-written



  1. a subclass of future
  2. wraps a coroutine
  3. when coroutine is finished, it returns result then task is finished




  1. For asynchronous programming, we need event loop
  2. We register our tasks/ futures in the event loop
  3. The event loop schedules them and executes them
  4. Callbacks are used so that we are notified when tasks/ future return results
  5. Coroutines are wrapped in tasks/ futures
    • when yield is finding, coroutine is paused
    • when yield gets a value, coroutine continues
  6. If coroutine doesnot return a value but returns an exception, then task fails.


This is a simple program i tried with Python’s asyncio module:

Screenshot from 2016-04-27 17-41-52


Python asyncio module



basically defines a coroutine


loop = asyncio.get_event_loop()

creates an event loop



runs  a loop until stop() method is called

Asynchronous programming

April Fedora Meetup, Pune

The Fedora meetup was conducted by Kushal Das on 23rd April 2016. Around 15 people attended the meetup at Kushal’s place.

In this meetup, Kushal introduced us to Unit Testing. Its a term often heard in Software development.



Testing is done to determine if there are errors in the code. It does not prove that the code is correct but it just checks that the conditions are handled correctly. Tests are as important as implementation itself.


Unit testing

Unit testing is done to evaluate a particular code component. These components can be function , class etc. It determines how well the component reacts to valid or invalid input.  Unit tests are written and executed by software developers.


unittest – an automated testing framework in Python

I installed the unittest module on my Fedora machine using:

pip install unittest

I used unittest module in my python program using:

import unittest


Basic Test Structure

There are two parts of a basic test:

  1. The code to prepare for test (test fixture)
  2. The code to test


Our first Test Case

This program tests cat command in Fedora. The cat command is used to read the contents of a file. We use the cat command in the following way:

Screenshot from 2016-04-25 14-47-30.png

This following program is the first test case. It prints:

  • the output of cat command
  • the error
  • the exit code



I saved the script with the name “”. The simply ran the script directly from the command line:

Screenshot from 2016-04-25 15-05-45

The above result shows:

  • The output of cat command.
  • The command ran successfully so error message is empty
  • exit code is “0”(successfully completed)

After that, there is a “.” before the dashed line above which indicates that the test passed.

Next, it shows the amount of time the tests took.

After that, the test outcome i.e. OK is shown.

Every test has 3 possible outcomes:

  • OK: the test passes
  • FAILED: the test doesnot pass  and raises “AssertionError” exception
  • ERROR: the test raises error other than “AssertionError”

Now, i did modification to the above program and changed the file name to something else i.e. gave a wrong file name and then ran the script. This is the result:

Screenshot from 2016-04-25 15-18-16

Here, since the file name is wrong, the cat command throws an error. The output is empty. The error message is displayed along with the exit code “1”(unsuccessful completion).

Kushal told us that the “b” before the output and error message indicates that it is in “bytes”.

To convert the output in string, we inserted the following line before print statement:

Screenshot from 2016-04-25 15-23-47



Assert statement is used for verifying expectations or some property of code being tested is true. Assert literally means “to state as fact”.

Assert statement stops the routine and sends the error message.

There are many assert statements like:

  • assertEqual
  • assertNotEqual
  • assertTrue
  • assertFalse
  • assertIn
  • assertNotIn
  • assertIs
  • assertIsNot

In the meetup, we tried “assertIn”. assertIn(a, b) checks if a is in b.

This following is the code modification that we made. Here, the assertIn checks if the word “fedora” is present in the output.

Screenshot from 2016-04-25 15-54-57.png

This is the result:

Screenshot from 2016-04-25 15-56-31

This means that the string “Fedora” is present in the output of cat command.

We modified the string “Fedora” to generate an error in this way:

Screenshot from 2016-04-25 15-58-45

After running the script, this is the result:

Screenshot from 2016-04-25 15-59-57.png

The test fails as the string “Fedra” is not present in the output.

Kushal, Chandan Kumar and Praveen Kumar gave all of us a simple task of testing.

My task was to:

  • change a file’s SELinux permissions to root (not through mode)
  • Write a test case to check if it gives an error when i try to access it through normal user sudo chmod 700
  • Check if the error message is present in the SELinux’s log file.


In this meetup i learnt about unit testing in Python; also I learnt about SELinux for the first time while solving the task.

Thanks to Kushal for providing the yummy snacks 🙂 and conducting the meetup.

April Fedora Meetup, Pune

Running fedocal instance


A web based calendar application for Fedora.  It is a single place where you can keep track of:

  • meetings
  • events
  • deadlines


Get the project

Fork the Fedocal project from Pagure.

Clone the project:


Then, go to the Fedocal directory:

Screenshot from 2016-04-18 19-14-45


Copy configuration files



There are 3 things to be done now:

1. Set Secret Key in configuration file

Install pwgen to generate 50 characters long random key:


Generate secret key:


Put the secret key in fedora.cfg file:

Screenshot from 2016-04-18 21-52-25


2. Adjust alembic configuration file:

go to /etc


make fedocal directory:


create alembic.ini file:


copy all data from alembic.ini in your cloned fedocal directory to this /etc/fedocal/alembic.ini

You can directly do this using “cp” command as well.

If you skip this step, then you might get this error while creating database scheme:



3.Set the database URL

In the alembic.ini file in your CLONED fedocal repo, you will find the line,

sqlalchemy.url = sqlite:////var/tmp/fedocal_dev.sqlite

In the fedocal.cfg file in you CLONED fedocal repo, you will find the line


Copy the sqlalchemy.url of the alembic.ini file to DB_URL of fedocal.cfg file as given below:


If, you skip this step and do not set the database URL, you get this error:



Install dependencies

Before creating database scheme, I had to install the following dependecies:


Even after installing python-openid, I was getting this error:


Now, according to this article, pycurl fetcher takes precedence if pycurl is installed. So, i checked if pycurl was installed on my machine using the command:


I removed pycurl using the command:


After that, I installed the remaining dependencies:



Create database scheme



Run the server

Screenshot from 2016-04-18 21-37-29.png


Now, access the server at http://localhost:5000 in your browser

Screenshot from 2016-04-18 21-40-08.png

Running fedocal instance

Running ircb instance

ircb is a irc bouncer. A bouncer connects to irc server and maintains a persistant connection. Instead of connecting to IRC server, connect to the bouncer. Even if you disconnect your client, it stays connected. If you log in after some time, it shows you the messages that you have missed.

You can find the source code of ircb here.

Following are the screenshots to show the steps i had done to run ircb instance:

Screenshot from 2016-04-17 18-43-31
Clone the repo


Screenshot from 2016-04-16 18-10-23
Install dependencies


Screenshot from 2016-04-16 18-15-03
Create virtual environment


Screenshot from 2016-04-16 18-16-59
Activate virtual environment


Screenshot from 2016-04-16 18-17-59
Install requirements


Screenshot from 2016-04-16 18-23-09
Create custom location directory


Screenshot from 2016-04-16 18-23-57
Create settings file in custom location


Screenshot from 2016-04-16 18-24-37
Copy settings


Screenshot from 2016-04-16 18-25-13
Export settings


Screenshot from 2016-04-16 18-25-39


Create a new user


Create network for that user


Run server
Open the endpoint using a IRC client.

Here I have used Hexchat which is a popular IRC client for Fedora. You can use xchat, irssi etc

I have used the most popular IRC network Freenode. Freenode listens on port 6667 for non-SSL and 6697 for SSL.


Add information for your bouncer

Servers of this IRC network: 6667

You can also add channels under “autojoin” tab.

Screenshot 1: connecting to server


Screenshot: Connected to server


Joining channel1 : #dgplug

The last step is to add some channels. Use the command “/join #<channel name>” to join a channel.

<channel name> is the name of a channel you want to join, such as #fedora, #dgplug, #waartaa.

Joining channel2: #waartaa

Now you never have to worry about missing another message in an IRC channel again!

Running ircb instance

Limit list size by slicing

While making the bugzilla widget in Fedora Hubs, I encountered a situation where i had to store only the required N numbers of item in a list. This could be done by using a while loop or an if condition but I learnt another way of doing this.


>>> country = []
>>> country = (country + [“india”])
>>> country = (country + [“pakistan”])
>>> country = (country + [“japan”])
>>> country
[‘india’, ‘pakistan’, ‘japan’]
>>> country = (country + [“australia”])[:2]
>>> country
[‘india’, ‘pakistan’]

This makes sure that the list size never exceeds 2.

Limit list size by slicing

My contributions in Fedora-Hubs

I have worked on 5 issues in fedora-hubs uptil now:

  1.  Pagure Issues Widget
  2.  Pagure Pull Request Widget
  3.  Github Issues Widget
  4.  Github Pull Request Widget
  5.  Bugzilla Widget

Fedora-Hubs source code:

The request module:

To extract information from Pagure, Github or Bugzilla, I used requests Python module.

Get more information on Request module here.

Install it using pip:

pip install requests

Steps involved

  1. Using requests module, extract the data from the api of website.
  2. Make sure you pass the appropriate username, project name in the URL.
  3. The return value of API calls is in JSON format.
  4. Now, simply using the basic Python slicing and indexing, extract the required data (e.g. id, issue/PR title) from the JSON data.
  5. Put this data (e.g. id, issue/PR title) appropriately in a data structure (I used a combination of list and dictionary)
  6. Return the data structure.

Pagure Widgets

This is the main page of Pagure:

This is the documentation of API of Pagure:

Example of a GET request URL:

  1. To list a project’s Issues:<name of the project>/issues
  2. To list a project’s Pull Requests:<name of the project>/pull-requests

These are the screenshots:

Pagure Issues Widget for Fedora-Hubs
Pagure Pull Requests Widget for Fedora-Hubs

Github Widgets

This is the documentation of API of Github:

Although there are specific github modules in Python. But since the issue involved only extraction and display of data, it can be done by using requests module itself.

Example of GET request URL:

  1.<name of the organisation>/<name of the repository>/issues
  2.<name of the organisation>/<name of the repository>/pulls


These are the screenshots:

Github Issues Widget for Fedora-Hubs
Github Pull Requests Widget for Fedora-Hubs

Bugzilla Widget

This is the Widget on which I am currently working on.

For this widget, I need one more module pkgwat.api module.

Get more information on pkgwat.api here.

Install it using pip:

pip install pkgwat.api

In this issue following steps occur:
1. Using requests module extract the packages of a specific user from fedora project api. This is the URL:<username&gt;

2. Pass each package name to pkgwat.api.bugs which returns all the issues of that particular package.

3. Store issue details(id, title) properly in a data structure.

4. Return the data structure

Screenshot from 2016-03-06 20-40-28
Bugzilla Issues Widget for Fedora-Hubs

Want to contribute to fedora-Hubs? checkout my blog on Fedora-Hubs.

My contributions in Fedora-Hubs




The Fedora.Next concept

This is where the idea of Fedora-Hubs evolved from. There are three components to this proposal:
1. Fedora “Brochure” Site
2. A Fedora User Support Site
3. A Fedora “Community Hub” —→ The idea of Fedora-Hubs

Read the whole idea about Fedora.Next here

The Fedora-Hubs

Fedora Hubs is a modern, web based Fedora activity center.

Need of Fedora-Hubs

1. Currently, the fedora web presence involves:
– mailing lists

2. It become daunting for new Fedora contributors to keep a track of various projects and teams they’re involved with in Fedora. Fedora-Hubs provides them a workflow.

3. Example:
Sometimes, new contributors have trouble using IRC because of the available chat systems they’re accustomed to. With the IRC widget of Fedora-Hubs, new contributors can use IRC without having to install anything.

4. It is like a social network for Fedora contributors.

5. Existing contributors can use it to collect their:
– ideas
– information
and keep a track of their activities.

Getting Started with Fedora-Hubs

The source code of Fedora Hubs resides in Pagure. You can find the source code here.

Fedora-Hubs uses Fedora Accounts System (FAS) to contribute as well as use. You can create a FAS acoount here.

Features in Fedora-Hubs

1. List of open tickets widget
2. General Activity Stream
3. Easy fixes

Checkout the issues page for a list of expected features.

IRC Channels

1. #fedora-apps:
Most of the discussion takes place here

2. #fedora-hubs:
a specific channel for Fedora Hubs

3. #fedora-design

Subscribe to mailing list here

Get more information from

1. Máirín Duffy:
IRC nick: mizmo
Twitter: @mairin

2. Meghan Richardson:
twitter: @meghanathema

3. Ryan Lerch:
IRC nick: ryanlerch

4. Ralph Bean:
IRC nick: threebean

5. Pierre Yves-Chibon:
IRC nick: pingou

6. Remy Decausemaker:
IRC nick: decause

Relevant Resources

1) Fedora-Hubs Wiki:

2) A collection of all the blogs on Fedora-Hubs by Máirín Duffy:

3) The blog of Meghan Richardson where you can get a lot of detailed information on Fedora-Hubs:

4) Post-Its on Fedora-Hubs by Ryan Lerch:

5) A news article on Fedora-Hubs:

6) A video of Máirín Duffy & Meghan Richardson on “Enabling new contributors with Fedora Hubs” in Flock 2015

7) A Brainstorm Session on “Enabling New Contributors in Fedora”:

8) A video on “Fedora Hubs Hackfest 2015”:

9) A PiratePad link to the discussion on Fedora-Hubs:

10) Links to all mockup resources:

a) The github source of all mockups:


Checkout “My contributions to Fedora-Hubs”.