Python Shell With Ipython
Aug 23rd, 2019 - written by Kimserey with .
The Python shell is a powerful tool to discover modules, test functionalities, and test our own application. IPython is an implementation of a python shell which provides enhancements over the original one. IPython is one of those programs that always have a space on my terminal tabs when building features for applications. In fact, it has become so important that it is now part of my workflow when working with Python. Today I want to share some of the functions and modules I use the most in order to iterate quickly on programs.
dir and help
Starting from the most basic functions, dir and help.
dir provides a list of names in scope. With no argument provided, it returns a list of the names in the global scope. With argument, it returns a list of the names in the scope of the argument provided.
1
2
3
4
5
6
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']
>>> import math
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'math']
We can see after importing math, it is now in global scope and we can then display the scope of math:
1
2
>>> dir(math)
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']
Using dir on classes will return the attributes of the class and of the base classes.
1
2
3
4
5
6
7
8
>>> class MyClass(object):
... def __init__(self):
... self._name = 'kim'
... def get_name(self):
... return self._name
...
>>> dir(MyClass)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'get_name']
While dir on an instance will return the class attributes, plus the instance attributes, like name in our example:
1
2
3
>>> c = MyClass()
>>> dir(c)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_name', 'get_name']
Another important build-in function is help. Help will print the help page attributed to the object provided as parameter.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
>>> help(math)
Help on module math:
NAME
math
MODULE REFERENCE
https://docs.python.org/3.7/library/math
The following documentation is automatically generated from the Python
source files. It may be incomplete, incorrect or include features that
are considered implementation detail and may vary between Python
implementations. When in doubt, consult the module reference at the
location listed above.
DESCRIPTION
This module provides access to the mathematical functions
defined by the C standard.
FUNCTIONS
acos(x, /)
inspect
Inspect is another nice module allowing us to inspect different aspect of a module, function or class.
1
>>> import inspect as i
Inspect.getmembers(object) is similar to dir, with the advantage that it provides the type of the member:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
>>> pprint([i for i in i.getmembers(math) if not i[0].startswith('_')])
[('acos', <built-in function acos>),
('acosh', <built-in function acosh>),
('asin', <built-in function asin>),
('asinh', <built-in function asinh>),
('atan', <built-in function atan>),
('atan2', <built-in function atan2>),
('atanh', <built-in function atanh>),
('ceil', <built-in function ceil>),
('copysign', <built-in function copysign>),
('cos', <built-in function cos>),
('cosh', <built-in function cosh>),
('degrees', <built-in function degrees>),
('e', 2.718281828459045),
('erf', <built-in function erf>),
('erfc', <built-in function erfc>),
('exp', <built-in function exp>),
('expm1', <built-in function expm1>),
('fabs', <built-in function fabs>),
('factorial', <built-in function factorial>),
('floor', <built-in function floor>),
('fmod', <built-in function fmod>),
('frexp', <built-in function frexp>),
('fsum', <built-in function fsum>),
...
We used getmembers to print the members which do not start with a leading underscore. With Python, all source files are accessible. This has the advantage of allowing anyone to look in more details on the underlying implementation. Inspect provides a function getmodule which allows us to know the exact module the object is coming from.
1
2
3
4
5
6
>>> import flask
>>> flask.app.request
<LocalProxy unbound>
>>> i.getmodule(flask.app.request)
<module 'werkzeug.local' from '/Users/kimsereylam/projects/blog/venv/lib/python3.7/site-packages/werkzeug/local.py'>
We can see that the object exposed by flask is a LocalProxy which can be found in Werkzeug code defined in /local.py.
1
>>> v = sqlalchemy.orm.query.Query.filter_by
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
>>> sl = inspect.getsourcelines(v)
>>> print(''.join(sl[0]))
def filter_by(self, **kwargs):
r"""apply the given filtering criterion to a copy
of this :class:`.Query`, using keyword expressions.
e.g.::
session.query(MyClass).filter_by(name = 'some name')
Multiple criteria may be specified as comma separated; the effect
is that they will be joined together using the :func:`.and_`
function::
session.query(MyClass).\
filter_by(name = 'some name', id = 5)
The keyword expressions are extracted from the primary
entity of the query, or the last entity that was the
target of a call to :meth:`.Query.join`.
.. seealso::
:meth:`.Query.filter` - filter on SQL expressions.
"""
clauses = [_entity_descriptor(self._joinpoint_zero(), key) == value
for key, value in kwargs.items()]
return self.filter(sql.and_(*clauses))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
>>> pprint([(name,type(getattr(sqlalchemy,name))) for name in dir(sqlalchemy)])
[('ARRAY', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('BIGINT', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('BINARY', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('BLANK_SCHEMA', <class 'sqlalchemy.util.langhelpers._symbol'>),
('BLOB', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('BOOLEAN', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('BigInteger', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('Binary', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('Boolean', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('CHAR', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('CLOB', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('CheckConstraint', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('Column', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('ColumnDefault', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('Constraint', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('DATE', <class 'sqlalchemy.sql.visitors.VisitableType'>),
('DATETIME', <class 'sqlalchemy.sql.visitors.VisitableType'>),
...
IPython Features
Magic Functions
The Python shell provides the bare minimum to explore and test code, in constrast IPython comes with a more feature complete set of functionalities.
So far we have seen dir, help and inspect, used to explore and test code, IPython comes with a set of magic functions provided the same functionalities and more.
For example, %pdoc will display the documentation just like help.
1
2
3
4
5
6
7
8
In [1]: %pdoc sqlalchemy.orm.query.Query.filter_by
Class docstring:
apply the given filtering criterion to a copy
of this :class:`.Query`, using keyword expressions.
e.g.::
session.query(MyClass).filter_by(name = 'some name')
%pdef will display the function definition and its signature:
1
2
In [2]: %pdef sqlalchemy.orm.query.Query.filter_by
sqlalchemy.orm.query.Query.filter_by(self, **kwargs)
But also question mark {object}? will give details on the object:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
In [1]: class X(object):
...: """ Test X class
...: Use this to make a test.
...: """
...: def __init__(self):
...: self.name = 'x'
...: def get_name(self):
...: return self.name
...:
In [2]: x?
Type: X
String form: <__main__.X object at 0x1058fa110>
Docstring:
Test X class
Use this to make a test.
Another great command from IPython is %run which allows us to run a python file from within the IPython Shell and dump all the content of the file in the global scope. When the file gets changed, we can just re-run it and the values will get updated.
1
In [8]: %run my_script.py
Laslty the debugger can also be switched on to jump to exception when they occur and get access to the pdb shell:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
In [39]: %pdb
Automatic pdb calling has been turned ON
In [40]: some_func(2, 'hey')
> <ipython-input-39-79f6dba91869>(4)some_func()
-> y=a+x
(Pdb) l
1 def some_func(a, b):
2 x=10
3 pdb.set_trace()
4 -> y=a+x
5 return '{} {}'.format(y, b)
6
[EOF]
(Pdb) dir()
['a', 'b', 'x']
(Pdb) c
Out[40]: '12 hey'
Indentation
Other than the magic functions, IPython also provide an automatic identation of the code which Python Shell doesn’t provide. Meaning that when creating classes or functions, it will automatically indent the next instruction allowing us to avoid inputting spaces ourselves.
1
2
In [14]: def some_func():
...: {starts here}
Another difference with the Python Shell is that IPython groups together in a clear manner the full command input and allows navigation back to input with up and down arrow. sIf we build the some_func definition, and press up, the whole definition will be suggested, in contrast to the Python shell which would only suggest line by line.
1
2
3
4
5
6
7
8
9
10
In [44]: def some_func():
...: print('Hello')
...: print('world')
...: return 10
...:
In [45]: some_func()
Hello
world
Out[45]: 10
Tab autocompletion
IPython also supports autocompletion via tabulation. After typing the beginning of a keyword, press tab and the autocompletion popup will appear.
History
The history functionality of IPython exposes the same behavior as the history of bash. CTRL+R can be used to search in history with suggestion, while repetitive CTRL+R will browse through the suggestions. Combined with the fact that IPython understands the composition of classes and definitions, it will suggest the whole implementations of classes which is very handy.
1
2
3
4
5
6
7
8
9
10
In [47]: class MySuperClass(object):
...: """ Test class
...: Use this to make a test.
...: """
...: def __init__(self):
...: self.name = 'x'
...: def get_name(self):
...: return self.name
...:
I-search backward: MySu
Another nice history management feature is provided by saving each input and ouput and making them available in the global scope respectively through _i and _.
_i has the previous input while _ contains the previous output. _ii will have the next previous input while __ will have the next previous output.
Input and ouput can also be targeted through their number, where _i13 will have the value of In [13] and _24 will have the value of Out [24].
IPython Startup
Lastly if we wish to run Python script prior the shell run, like how we would do settings the PYTHONSTARTUP, we can put them in under ~/.ipython/profile_default/startup/. Scripts are ran in alphabetical order therefore the README added by default advises to prefix scripts with a number to control the order.
1
2
3
00-init.py
10-other-script.py
20-other-script.py
And this concludes today’s post.
Conclusion
Today we saw how we could explore and quickly code Python scripts using the Python shell. We started by looking at two of the fundamental functions to discover features, dir and help. We then looked into inspect which provides more advance features to dig deeper in a codebase. Lastly we looked at another implementation of the Python Shell called IPython which comes equipped with a lot of good functionalities like history management, autocompletion, and even magic functions making it very easy to work with Python within the shell. I hope you liked this post and I see you on the next one!