Using _ (underscore) as a placeholder or temporary variable name

The underscore character is often been used as a placeholder or throwaway variable in python. Like the following snippet:

for _ in range(10):
    print("Hello World")

Here, as we do not use the variable needed for iterating range,it's replaced with the `_` (underscore)  character.

In python there are three main usage of  `_` (underscore) character. (Taking reference from a well viewed Stack Overflow answer)

• To hold the result of the last executed statement in an interactive interpreter session. In the following snippet we did not assign the return value of the split()method. But we can access the value using the `_` character.

In [1]: "Hello World".split()
Out[1]: ['Hello', 'World']

In [2]: _
Out[2]: ['Hello', 'World']

• For translation lookup in i18n. Like the following often seen python web based projects.

forms.ValidationError(_("Please enter a correct username"))

• As a general purpose "throwaway" variable name to indicate that part of a function result is being deliberately ignored. Here get_product will returns 3 value but we are only interested in the first two.

product_name, product_code, _ = get_product(product_id)

If we need a throwaway or placeholder variable, it's always a good practice to use the underscore character. Using a named variable in such case can create confusion. Using the `_` (underscore) characters in these circumstance will tell the reader that the data is unnecessary.

Avoiding direct comparison with None or True, False

Comparing Bool type objects or None

Consider the following snippets:
snippet 1

if doc is None:
    doc = {"key": "value"}

snippet 2

if not doc:
    doc = {"key": "value"}

I have seen both types of code several times. Now the obvious question is which one is more "pythonic" and why.

Here the second approach is the "pythonic" way to do it. We know that the is keyword determines if the comparing objects are actually same objects or not. So if the doc is any other objects that are considered False in python then the code will not work as it was supposed to. But the second code will perform for any thing that is considered False in python.

Things that are considered False in python are:

• False
• None
• 0 (Numeric)
• [] (empty list)
• {} (empty dictionary)
• if  __len__() or __nonzero__() return False or 0

So unless it is necessary to compare directly with any of these objects, snippet 2 type of approach should be followed. 

GNU Screen how to

Overview

screen (GNU screen) is a software which is used to create virtual terminals. It can run on it's own when not under direct user control. This is especially useful while working with a remote system, when a disconnect can occur at any moment. In this case, the screen session will continue to execute and you may reattach a running screen later. 

I found it very useful in the cases where I need to run a script in a remote server which takes long time time to execute.

Installation

In a Debian based operating system it can be installed by this command:

sudo apt-get install screen

Basic Usage

Starting a new screen session:

screen -S <screen_name>

This command will start a new screen session. After running the process, the screen can be detached by pressing:

ctrl + a + d

All available screen can be viewed using this command:

screen -ls

Reattaching a screen can be performed by typing:

screen -r <screen_name>

REST Api with Flask-Restful and MongoDB

What is REST

REST (Representational State Transfer) is an architecture style for designing networked applications. The idea is using simple HTTP methods for making calls between machines rather than complex mechanisms such as CORBA, RPC or SOAP. 

So a RESTful system typically communicates over the Hypertext Transfer Protocol with the same HTTP verbs (GET, POST, PUT, DELETE, etc.)

Flask and Flask-RESTful

Flask is a microframework for Python based on Werkzeug, Jinja2. Flask-RESTful provides an extension to Flask for building REST APIs.

Installing packages

For the api, we need to install some packages like Flask-Restful, pymongo etc. It's good practice to install packages and modules for a project in a virtual environment. So i am going to create a virtual environment and activate it.

$ virtualenv env

$ source env/bin/activate

Now the required packages can be installed using pip. (Link: requirements.txt)

$ pip install -r requirements.txt

Api

Lets Code the API first.

This is a basic api with only one Resource Student. First we created an instance of Flask,
app = Flask(__name__)
after that we configured app with MongoDB.

The resource Student has HTTP methods defined in it as it's own method. By calling different HTTP methods we can perform basic CRUD operations. Like post for inserting/creating, put for updating, get for reading and delete for deleting a document.

In the code post, put, delete methods are basic.  These methods perform create, update and delete operation on the collection student. 

But in the get method, by calling different url we can perform different operations. If no parameter passed with url, the method will return all the documents in the collection. If a registration number passed with the url then, it will check the parameter name and will return one document. If request is made in /api/department/<string:department> url, it will check for parameter name department in the get method and returns all the documents for a particular department.

There are several urls for same resource so we added different endpoints for them.

Testing the Api

We will use python requests module (version > 2.4.2) for testing the api. But also curl and other mechanism can be used. For get request we can view the data in browsers also.

requests module gives us convenient way to test our api. Enough REST. Lets take some rest ;)

PYTHONPATH and importing modules in python

What is PYTHONPATH

PYTHONPATH is an environment variable that defines additional locations of Python modules. This is the default search path for module files. This means that python interpreter will search for modules in the location of the PYTHONPATH.

How importing works in python

When a module named mymodule is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named mymodule.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

• The directory containing the input script (or the current directory).
• PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
• the installation-dependent default.

problem scenario and solve:

A

|--__init__.py

|

|--B

|--|--__init__.py

|--|--greetings.py

|

|--C

|--|--__init_.py

|--|--mymodule.py

Here A is a python package. Inside A, there are two packages B and C. 

greetings.py

def my_function():
    print "Congrats Bangladesh Cricket Team!! :D"

mymodule.py

from B.greetings import my_function
my_function()

Now, there is a problem. Package A  is not in python interpreter's search path. So it can not import greetings function from B  within C. If mymodule.py is run, it will raise an ImportError as expected.

$ python mymodule.py

Traceback (most recent call last):

  File "mymodule.py", line 3, in <module>

    from B.greetings import my_function

ImportError: No module named A.B.greetings

To solve it, we need to add the absolute path of package A in sys.path inside mymodule.py before importing greetings from B. Path will be appended in sys.path. This will not affect other python program because the path will not be modified for any other Python processes. 

mymodule.py

import os
import sys

PATH = os.path.abspath(os.path.join(os.pardir))
sys.path.append(PATH)

from B.greetings import my_function
my_function()

Now as the path of A is added, Python interpreter can find the package B inside A. Let's run again this function

$ python mymodule.py

Congrats Bangladesh Cricket Team!! :D

MongoDB model with PyMongo

MongoDB

MongoDB is a popular open-source NoSQL database. NoSQL is a database technology to store and retrieve data which are not stored in database like the traditional tabular form of RDBMS. Scalability and superior performance are main beneifts of NoSQL databases over RDBMS. Mongodb uses JSON-style documents with dynamic schema to store data. Installation instructions of MongoDB will be found in it's documentation - Install MongoDB. 

PyMongo

PyMongo is a Python distribution containing tools for working with MongoDB, and is the recommended way to work with MongoDB from Python. To install PyMongo execute the command in terminal:

$ sudo pip install pymongo

Creating a model

I am going to create student model which will be used to  get, create, update, delete objects form database.

models.py

# -*- coding: utf-8 -*-
import pymongo


class Student(object):
    def __init__(self, registration, **kwargs):
        self._id = registration
        self.first_name = kwargs.get('first_name', '')
        self.last_name = kwargs.get('last_name', '')
        self.department = kwargs.get('department', '')
        self.session = kwargs.get('session', '')
        self.mobile = kwargs.get('mobile', '')

    def __str__(self):
        return self._id

    @staticmethod
    def get_db(dbname='student_db'):
        """
        make database connection and return a database instance
        """
        client = pymongo.MongoClient(host='localhost', port=27017)
        return client[dbname]

    @classmethod
    def get(cls, registration):
        """
        get student by registration number
        """
        db = Student.get_db()
        student_info = db.students.find_one({'_id': registration}, {'_id': 0})
        if student_info:
            student = Student(registration=registration, **student_info)
        else:
            student = Student(registration=registration)
        db.connection.close()
        return student

    @classmethod
    def get_all(cls):
        """
        get all students and return a list. for many student return a cursor instead.
        """
        db = Student.get_db()
        students = list(db.students.find())
        db.connection.close()
        return students

    def save(self):
        """
        save() method will create or update a document based on the _id attribute.
        """
        db = Student.get_db()
        db.students.save(self.__dict__)
        db.connection.close()

    @classmethod
    def remove_all(cls):
        """
        remove all documents for the collection.
        """
        db = Student.get_db()
        db.students.remove()
        students = list(db.students.find())
        db.connection.close()
        return students

    def remove(self):
        """
        remove a object from the collection.
        """
        db = Student.get_db()
        db.students.remove(self._id)
        db.connection.close()
        return self._id

Now this model can be used for CRUD operations.

>>>from models import Student
>>>student = Student.get('2008331041')
>>>student.first_name = "Salman"
>>>student.last_name = "Wahed"

>>>student.save()
>>>Student.get_all()

[{u'_id': u'2008331041',
  u'department': u'',
  u'first_name': u'Salman',
  u'last_name': u'Wahed',
  u'mobile': u'',
  u'session': u''}]

Unit testing in python using unittest framework

What is Unit test

Unit test is a process of testing an unit of testable code like functions, classes, procedures etc of an application using test code. Mainly programmers write unit tests for their codes while developing software. Unit testing reflects the programmers view on how things should work. This is the main theme of TDD (Test-driven development).

Unit test in python

In python there are some frameworks to do unit testing  like unittest, nose, pytest etc. Among these, unittest module comes with default python installation.

I have written a function which will take an integer value as argument and return if that integer is even or odd.

pytesting/even_odd.py

# -*-coding: utf-8-*-
def my_even_odd(number):
    if isinstance(number, int):
        if number % 2 == 0:
            return "Even"
        return "Odd"    
    return "Integer required"

This function will return "Integer required" if it gets an argument other than int. And for each argument it should return a string. Here, I am going to test two things: one is if I pass an integer or a non integer argument what it will return and  two, if it returns anything other than string.

pytesting/unittest_even_odd.py

# -*-coding: utf-8-*-
import unittest
from even_odd import my_even_odd

test_values = [
    (1, "Odd"),    (0, "Even"),    (2147483648, "Integer required"),    (2147483647, "Odd"),    (-2147483649, "Integer required"),    (-2147483648, "Even"),    ('', "Integer required"),    ({'a': 1}, "Integer required"),    ([2, 3, 5], "Integer required"),]


class TestMyEvenOdd(unittest.TestCase):
    def test_my_even_odd(self):
        for tpl in test_values:
            self.assertEqual(my_even_odd(tpl[0]), tpl[1], msg="Failed for {}".format(tpl[0]))
            self.assertTrue(isinstance(my_even_odd(tpl[0]), str))


if __name__ == '__main__':
    unittest.main()

In unittest framework to create a test we need to create a subclass of unittest.TestCase class and add or override appropriate methods. In this case I added a method test_my_even_odd. There are two main parts. One is the test class we have written and another is test fixture. A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions. This may involve, for example, creating temporary or proxy databases, directories, or starting a server process etc. In our case we keep it simple by using a list test_values. assertEqual and and assertTrue methods of  a TestCase class.

assertEqual(self, first, second, msg=None)

This method will raise an AssertionError and test will fail if first and second objects are unequal as determined by `==`

assertTrue(self, expr, msg=None)

This method will check if the expression expr is True and raises AssertionError if False.

Now if we run unittest_even_odd.py we will get output like this,

$ python unittest_even_odd.py

.
--------------------------------------------------------------------
Ran 1 test in 0.000s

OK

That means my little function passed the test.