• Setting up a Debian VM with Vagrant on Windows

    As a developer I endeavor to keep a consistent working environment on all of my computers. Meaning editing with VIM, version-controlling with GIT and using all the cool POSIX tools directly from the Terminal to do whatever. At home I use Mac OS X and for servers I use Debian. So working with the terminal is basically the same for both systems.

    Unfortunately on Windows it is not as easy to setup a proper toolchain as it is not a Posix compliant operating system. So instead of a powerful shell like Bash with lots of helper tools you get the Commandline with almost nothing. Luckily there is Cygwin which brings some kind of Linux-flaire to Windows by providing Posix tools and a nice terminal called mintty. Still it is not the same as running on Linux but it is a step in the right direction. The main drawback using Cygwin is that you are still on Windows. Meaning you can't just clone a Github repository and compile it. When you do so there is always something missing. Also you keep running into trouble with Cygwins way of putting all drives in /cygdrive instead of using the native drive letters on Windows. So to get the real Linux feeling I decided to actually use real Linux ... in a virtual machine.

    VM with Vagrant

    Vagrant is great for easily setting up virtual environments readily configured and with all tools installed that you prefer for your development activities. Before getting started we need to install Vagrant and a virtual machine of our choice. I actually use Oracles VirtualBox. Once done we can set up our development machine. We start mintty and create a directory vagrant where we enter two commands.

    $ vagrant init debian/jessie64
    $ vagrant up

    The first line downloads us a fresh Debian Jessie box and the second line starts it up. Now we can ssh into it by calling:

    $ vagrant ssh

    Great. A fully working Debian set up with just two commands. But that is not all we want. We want our develpment environment to be already provisioned with all our favorite development tools. And that is where Vagrant is really scoring. We can provide a provisioning shell script where we install all our needed tools. Leave the virtual environment and open up the freshly created Vagrantfile. Put these few lines at the end of the file.


      # ...
      config.vm.provision "shell" do |s|
          s.binary = true
          s.path = "provision.sh"

    What it does is tell Vagrant to use Windows line endings in your VM which is important if you edit files located on your Windows system. Windows line-endings look like \CR\LF while Unix line-endings only use \LF. If you don't set s.binary = true you will run into issues with the different line-endings. Also we will set a provisioning script that will install our toolchain.


    sudo apt-get -y update
    sudo apt-get -y upgrade
    echo -e "\n--- Install dev tools ---\n"
    sudo apt-get -y install mc htop vim git
    # set Windows lineendings for git
    git config --global core.autocrlf true
    echo -e "\n--- Clone vim settings ---\n"
    cd /home/vagrant
    git clone https://github.com/MrLeeh/.vim.git
    ln -s .vim/.vimrc .vimrc
    echo -e "\n--- Install Python3 ---\n"
    sudo apt-get -y install python3 python3-pip

    What it does:

    • update the package database and apply upgrades to all installed Linux packages
    • install my most needed dev tools: mc, htop, vim, git
    • set Windows line-endings for git
    • clone my .vim directory from Github and create a link to .vimrc so I get all my favorite VIM settings right from the start
    • install python3 and pip

    So all together we get a nice development environment that we can easily put on every computer we want to work with. The last thing that's missing is access to our files located on the host machine from our virtual environment. For this we only need to modify our Vagrantfile again and add all the directories we wish to sync.


    config.vm.synced_folder "C:/Users/johndoe/documents", "/home/johndoe"

    Now you can access all files in c:\Users\johndoe\documents from our VMs /home/johndoe and use our nice Linux toolchain on them. So we're done. Have fun...

  • Flask-Admin: Handle image selection

    In my last post Checking out: Flask-Admin extension I gave a short introduction to the Flask-Admin extension. I also built a small example to show how easy it is to get a basic admin interface for your data. But how does Flask-Admin work if we have more advanced requirements. For example what if we want to provide an image for each user. In this case we will have to expand our recent example by an Image model.

    class Image(db.Model):
        id = db.Column(db.Integer, primary_key=True)
        name = db.Column(db.String(128))
        path = db.Column(db.String, unique=True)
        def __repr__(self):
            return self.name

    To administer the new Image model we need to create a ModelView for it. This time it won' t be enough to just use the ModelView provided by Flask-admin. We want to be able to upload a new image via an upload button and get a nice preview of our image. We need to create our own ImageView class here. I basically used the Flask-Admin example Files, images & custom forms here.

    basedir = os.path.abspath(os.path.dirname(__file__))
    file_path = os.path.join(basedir, 'files')

    Alright, that' s it. You can find the whole example on https://github.com/MrLeeh/flask-admin-examples-images.

    class ImageView(ModelView):
        def _list_thumbnail(view, context, model, name):
            if not model.path:
                return ''
            return Markup(
                '<img src="%s">' %
        column_formatters = {
            'path': _list_thumbnail
        form_extra_fields = {
            'path': form.ImageUploadField(
                'Image', base_path=file_path, thumbnail_size=(100, 100, True))
    admin.add_view(ImageView(Image, db.session))

    That's great, we can upload images. Now we want to use them to give our users a face. For this we need to modify our User model and create a UserView class.

    class User(db.Model):
        id = db.Column(db.Integer, primary_key=True)
        username = db.Column(db.String(64), unique=True)
        image_id = db.Column(db.Integer, db.ForeignKey('image.id'))
        image = db.relationship('Image')
    class UserView(ModelView):
        def _list_thumbnail(view, context, model, name):
            if not model.image or not model.image.path:
                return ''
            return Markup(
                '<img src="%s">' %
        column_formatters = {
            'image': _list_thumbnail
    admin.add_view(UserView(User, db.session))

    Now this will show the image for each user in the user list. That is very nice. Also when we edit a user we magically get a nice Select2 field that let's us choose our user image. Only we don't get a preview of this image. To include a preview we need to provide our own edit template edit_user.html:

    {% extends 'admin/model/edit.html' %}
    {% block head %}
    {{ super() }}
    <script   src="https://code.jquery.com/jquery-2.2.4.min.js"   integrity="sha256-BbhdlvQf/xTY9gja0Dq3HiwQF8LaCRTXxZKRutelT44="   crossorigin="anonymous"></script>
      $(document).ready(function() {
        $('select').on('change', function(event) {
          var img_id = $(this).val();
            '{{ url_for('_get_image_url') }}',
            {img_id: img_id}, 
            function(data) {
              if (data.status == 'ok') {
                $('img').prop('src', {{ url_for('static', filename='') }} + data.img_path);
              else {
    {% endblock %}
    {% block edit_form %}
    {{ super() }}
    <img src="{{ url_for('static', filename=model.image.path) }}" style="max-height:200px;">
    {% endblock %}

    We tell Flask-Admin to use this template by adding the following line to our UserView class .

    edit_template = 'edit_user.html'

    Now what is it doing. Actually we keep the default template but only add an image tag at the bottom. We use jQuery to listen to the change event of our image select field. If it is changed we will send an ajax call to our server, ask for the image path of the currently selected image and load it in our img element.

    Of course for this to work we still need to implement the endpoint get_image_url in our application.

    def _get_image_url():
        img_id = request.args.get('img_id')
        img = Image.query.get(img_id)
        if img is None:
            response = jsonify(status='not found')
            return response
        return jsonify(img_path=img.path, status='ok')

    Alright, that' s it. Now you can choose and preview the images of your user. You can find the whole example on https://github.com/MrLeeh/flask-admin-examples-images.


  • Checking out: Flask-Admin extension

    In my recent projects I often ran into the important but boring task of building admin interfaces on top of data models. It is important because using an admin interface it is the preferred way to edit, add and delete information. It is boring because it basically means you have to create two interfaces dealing with exactly the same data: the layout of the website that is shown to public and the interface for data administration. Flask-Admin is an addon for the Flask microframework that takes care of the boring part. With little effort, it lets you manage your web service’s data through a user-friendly interface.

    A basic example

    My first shot was to try a basic example for editing the user table of a Flask application. The User class is a simple model containing just the id and username. I use the great Flask-SQLAlchemy extension here.

    import os
    from flask import Flask
    from flask_admin import Admin
    from flask.ext.sqlalchemy import SQLAlchemy
    from flask_admin.contrib.sqla import ModelView
    basedir = os.path.abspath(os.path.dirname(__file__))
    app = Flask(__name__)
    app.config['SQLALCHEMY_DATABASE_URI'] = (
        'sqlite:///' + os.path.join(basedir, 'data-dev.sqlite'))
    app.config['SECRET_KEY'] = 'my secret'
    db = SQLAlchemy(app)
    admin = Admin(app, name='microblog', template_mode='bootstrap3')
    class User(db.Model):
        __tablename__ = 'users'
        id = db.Column(db.Integer, primary_key=True)
        username = db.Column(db.String(64))
    # Add administrative views here
    admin.add_view(ModelView(User, db.session))

    The admin interface is added by simply creating an instance of the Admin class. A model view is added by calling the add_view() function on the admin object. Run the example above and visit http://localhost:5000/admin and you will see the new admin interface:


    When clicking on one of the entries you get a nice detail view that allows you to edit the available model data:


    I find this super-easy. We can now add, edit and delete users in a very convenient way. Some more advanced examples can be found onhttp://examples.flask-admin.org/.

  • Serve a Flask application on a servers subdirectory

    Lately I wanted to test a new application before deployment and wanted to make it accessible via a subdirectory of my blog url. It doesn't sound like a great deal but I had some difficulties to accomplish it. To make my life easier in future and to help others facing the same challenge I wrote this Howto.

    The Application

    For demonstration purposes we'll create a small Flask app. An important issue when you put a WSGI application in a subdirectory is about pathes and filenames. So we'll create an app that uses a template and a static file. The application structure looks like this:

    • flaskapp
      • app
        • __init__.py
        • views.py
        • static
          • python.png
        • templates
          • index.html
      • run.py


    from flask import Flask
    app = Flask(__name__)
    from . import views


    from flask import render_template
    from . import app
    def index():
        return render_template('index.html')


    <h1>Welcome to App1</h1>
    Image "{{ url_for('static', filename='python.png') }}":<br/>
    <img src="{{ url_for('static', filename='python.png') }}">

    Originally the app is served from http://servername.com so all Flask routes and also static file pathes will be interpreted relatively to the root. Flask will take care of this automatically if you use the url_for function to create links in your templates and the route decorator in your views module. But If you serve from a subdirectory you have to tell Flask that your root is a different one. So our {{ url_for() }} function in index.html should deliver http://servername.com/app1/static/python.png instead of the original http://servername.com/static/python.png. We accomblish this by using a middleware. For running our application we will create the file flaskapp/run.py


    from werkzeug.wsgi import DispatcherMiddleware
    from werkzeug.serving import run_simple
    from app import app
    application = DispatcherMiddleware(
        None, {
            '/app1': app
    if __name__ == '__main__':
        run_simple('localhost', 5000, application, use_reloader=True)

    The werkzeug.wsgi.DispatcherMiddleware class allows one to mount middlewares or applications in a WSGI application. This is useful if you want to combine multiple WSGI applications. And that is what we want to do here. We don't have a main application for the server root directory so we will provide None as first argument. The second argument is a dictionary containing the url subdirectories as keys and the corresponding WSGI objects as values. Now we can run our Flask application by calling:

    $ python run.py

    We visit http://localhost:5000/app1/ and find a our small Python logo facing us. Success :)

    Reverse proxy

    I use nginx as a reverse proxy. So for my new app to work I created a new configuration file /etc/nginx/sites-available/app1.

    upstream app1 {                                                                                                 
    server {                                                                       
        listen *:80;                                                               
        root /home/app1/flaskapp1/app;
        location /app1/static {                                                    
            alias /home/app1/flaskapp1/app/static;                                 
        location /app1 {                                                           
            proxy_pass http://app1;                                                

    It basically passes all calls to /app1 on to the WSGI application and serves static files separately from the static directory. Put a symbolical link to this file in /etc/nginx/sites-enabled to enable it and restart nginx.

    $ service nginx restart


  • Flask Webserver Deployment - Part 4: Nginx Proxy Server

    This is the final part of my tutorial on Flask Server Deployment. In the last section we configured our Gunicorn WSGI server and controlled it via Supervisor. In this section we will place our WSGI server behind a Proxy server.

    Content of this Tutorial

    Why use a Proxy Server

    This is the question I asked me first when I read in the Gunicorn documentation that one should put the Gunicorn application server behind a proxy. Mainly the main advantages of a proxy seem to be:

    • you can run more than one WSGI server with different web applications and assign them to individual routes or ports
    • the proxy takes care of static content while the WSGI server handles the dynamic stuff
    • it frees your Gunicorn server from serving slow clients

    The guys from Gunicorn recommend Nginx as a proxy so I decided to follow this recommendation. One could also use Apache here.

    Install and configure Nginx

    We install Nginx via apt-get:

    $ apt-get install nginx

    The configuration happens mainly via the /etc/nginx/nginx.conf file. We open it with our editor and add our server configuration:

    server {
        listen 80;
        server_name _;
        access_log  /var/log/nginx/flaskapp.log;
        error_log  /var/log/nginx/flaskapp_err.log;
        location / {
            proxy_redirect     off;
            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

    What we do now is listen on port 80 for any requests. Those requests are passed on to port 8000 of our localhost. This is where our Gunicorn server will serve our flasktest app. The proxy_set_header directives are used to rewrite the headers so that our application works behind a proxy. Before we start of with our new proxy configuration make sure you changed the port in our Gunicorn startup script /home/apps/run_flaskapp.py from 80 to 8000 and restart it with

    $ supervisorctl restart flasktest

    Now we restart our nginx server:

    $ service nginx restart

    Now we will see our "Hello World" statement served from a WSGI application server behind a proxy.

    Useful Links

  • Flask Webserver Deployment - Part 3: Gunicorn WSGI Server

    This is Part 3 of my tutorial for Flask Webserver Deployment. In the last section we prepared our server for serving a Flask application. We created a small Flask test application and finally started up the Flask development server. In this part we will use a more production ready server called Gunicorn to run the application.

    Content of this Tutorial



    Gunincorn 'Green Unicorn' is a Python WSGI HTTP Server for Unix. It is ported from Ruby's Unicorn project and it is supposed to be "simply implemented", "light on server resources" and "fairly speedy".

    We can easily install Gunicorn via Pythons package installer pip. Make sure to change to the virtual environment we created in the last part of the tutorial and install it:

    $ cd /home/apps
    $ source /venv/bin/activate
    (venv) $ pip install gunicorn

    WSGI Script

    Gunicorn expects a Python module with a WSGI object in it. So we create an new Python module named wsgi.py containing the following line of code:

    from app import app as application

    This module only opens up our app module and imports Flasks WSGI app object.

    Now we can start our server by calling:

    (venv) $ gunicorn wsgi -b127.0.0.1:80
    [2016-04-28 23:22:43 +0000] [1476] [INFO] Starting gunicorn 19.4.5
    [2016-04-28 23:22:43 +0000] [1476] [INFO] Listening at: (1476)
    [2016-04-28 23:22:43 +0000] [1476] [INFO] Using worker: sync
    [2016-04-28 23:22:43 +0000] [1479] [INFO] Booting worker with pid: 1479



    We got our Gunicorn server working but it is a bit inconvenient. We need to manually activate our virtual environment and start up the Gunicorn server. Actually we want our server to startup automatically. To achieve this we will use a process control system called supervisor. It is a Python2 application so in order to use it we will leave our virtual environment. Make sure you have installed the Python2 packages python and python-pip.

    (venv) $ deactivate
    $ apt-get install python python-pip
    $ pip install supervisor


    We need a startup script that does the following things for us:

    1. activate our virtual environment
    2. start the Gunicorn server

    We create a new file called run_flaskapp.py in /home/apps with the following lines of code:

    # activate virtualenv
    cd /home/apps
    source venv/bin/activate
    # run gunicorn
    exec gunicorn wsgi -b0.0.0.0:80

    Supervisor comes with a config template which we will copy in our actual supervisord.conf file.

    $ echo_supervisord_conf > /etc/supervisord.conf
    $ vim /etc/supervisord.conf

    We add our flasktest app here by adding the following text at the bottom of /etc/supervisord.conf:

    [program: flasktest]

    Run it

    We start the supervisor daemon by calling:

    $ supervisord

    This should automatically startup our flasktest app. Now we can use supervisorctl to start and stop our application:

    $ supervisorctl start flasktest
    flasktest: started
    $ supervisorctl status flasktest
    flasktest                           RUNNING   pid 1504, uptime 0:00:49
    $ supervisorctl stop flasktest
    flasktest: stopped


    To automatically startup the supervisord daemon and with it our application on system start we need an init script. Luckily somebody had the idea of writing it for us and putting it on GIT. We can find init scripts for different Linux distributions on https://github.com/Supervisor/initscripts.git.

    In my next post we will put our Gunicorn application server behind a Nginx proxy server for increased savety reasons.

  • Flask Webserver Deployment - Part 2: Flask Development Server

    This is Part 2 of my tutorial for Flask Webserver Deployment. In the last section we acquired our own personal Linux server. Now we want to get connected to it and run our first Flask Test Application.

    Content of this Tutorial

    Preparing the Server

    Get connected

    Via the operating panel of our service provider we chose to use Linux Debian "Jessie" as our operating system. Now we make contact by using the Secure Shell (SSH). On a unix-like OS we can connect to the VServer by calling the following command:

    $ ssh root@hostname_or_ip

    We want to connect as root so we are able to modify our installed software. The server will answer by asking for the password and then gracefully present us its shell. The first thing we want to do is update our installed software packages:

    $ apt-get update
    $ apt-get upgrade

    ! You need root privileges for these commands. If you are not logged in as root use sudo here.

    Install Python and git

    Now all installed packages are up-to-date and we can start installing our framework. As I am a Python3 kind of person I will describe how to deploy a Python3 Flask Application. There is still an ongoing debate about whether to use Python2 or Python3. Just for the record: I prefer Python3 very much over Python2 mainly because all Strings are unicode now and I don' t have to worry about encoding anymore. Also most packages now support Python3 and I didn' t have any compatibility issues with Flask or any of its extension packages so far. If you chose Python2 just install the equivalent packages. So what do we need: git: The "flasky" application is hosted on github so we need git to deploy it to our server. python3 and python3-pip for quick and easy package installation

    $ apt-get install -y git python3 python3-pip

    Add a new user

    We don't want our application to run with root privileges. So we create a new user apps that will run it.

    $ adduser apps
    Adding user `apps' ...
    Adding new group `apps' (1001) ...
    Adding new user `apps' (1001) with group `apps' ...
    Creating home directory `/home/apps' ...
    Copying files from `/etc/skel' ...
    Enter new UNIX password: 
    Retype new UNIX password: 
    passwd: password updated successfully
    Changing the user information for apps
    Enter the new value, or press ENTER for the default
        Full Name []: 
        Room Number []: 
        Work Phone []: 
        Home Phone []: 
        Other []: 
    Is the information correct? [Y/n]

    This command creates the new user apps in the new user group apps. That's what we want. We are asked to choose a password and confirm it. Afterwards we are asked for some further information. We take take default by pressing ENTER.

    Create the Flask Test Application

    Let's change into the new users home directory and start up our favorite editor.

    $ cd /home/apps
    $ vim flasktest.py

    We create our sample application in a new file called flasktest.py.

    from flask import Flask
    app = Flask(__name__)
    def index():
        return 'Hello World'
    if __name__ == '__main__':
        app.run(host='', port=80)

    I know this is a bit boring but it serves our purpose.

    Install Python Packages in a Virtual Environment

    A Flask application typically has a number of Python packages it depends on. Normally that is of course Flask and some Flask extensions like Flask-Shell or Flask-SQLAlchemy. It is best practice to keep those package dependencies in a requirements file named requirements.txt. Our requirements.txt will just contain the Flask package so it looks like:


    We want our Python application to run in a virtual environment. This way it is isolated from the global Python packages and is easy to run multiple Flask application on one server. For this we install the virtualenv package.

    $ pip3 install virtualenv

    We create a new virtual environment and activate it. Then we install our required packages.

    $ virtualenv venv
    $ source venv/bin/activate
    (venv) $ pip install -r requirements.txt

    Run the development server

    As a first shot we want to use the Flask development webserver to serve our application. It is not a performant webserver and it should not be used for production. But we just want to know if all packages are installed correctly and the application is running. We start our Webserver by calling:

    $ sudo python flasktest.py

    We need root privileges to bind our server to port 80, so I use sudo here. Alright, we got it working. You can access the incredible 'Hello World' via your browser by calling your server ip-address. In my next post we will serve our application with a much more performant application server called gunicorn.

  • Flask Webserver Deployment - Part 1: Hosting Services

    Hello World again. This is my second post and I'm going to write about the steps it took me to get my Flask Application deployed to a Webserver.

    Content of this Tutorial

    Finding the right Hosting Service

    Of course when developing your web application there will be the time when you want to unleash it to the world. That means you need some kind of Webserver where you can deploy your application. There exist a huge amount of hosting services in the web. Apart from the classic (virtual) Linux server there are the so called Platforms as a Service (e.g. Heroku). A Platform as a Service allows you to deploy your app without worrying about the operating system and server configuration. You just write your application plus a small config file and upload it to the platform (e.g by pushing your git repository to the server). However you have to pay for this convenience. In fact Heroku provides a free account for experimenting but the app will got to sleep for at least 6 hours a day. For small scale projects without downtime you will have to pay 7$ a month.

    The cheaper but more sophisticated solution is to use a virtual Linux server and configure it the way that it serves your Flask application. Eventually one has to decide whether to save time or money. In my case I wanted to keep the monthly costs for my Blog low. Also I was curious about how to get my own Webserver running from scratch. So I will continue by describing how I got my Server working.


    If you want to serve a Flask application your webserver will have to provide at least the following features:

    • some kind of remote control, e.g. SSH
    • root privileges to install Python and Python packages your application depends on
    • depending on the kind of application you also need a Database server (mysql, mariadb)

    If you want your new application to be accessible by a Domain name and not only by the IP address of the server you will also need to acquire a domain name.

    Setting up the Server

    Let's get to work. In my next post I will describe how to set up a Linux Server with Debian "Jessie" installed on it in a way that it can be used to serve a Flask application. We will build a Flask Test Application and serve it with the builtin development server.

  • My first Post

    Hello World

    Hello, my name is Stefan and I wrote this small blogging platform in Python using its great Flask microframework. I was guided along the way by the very recommendable book "Flask Web Development" from Miguel Grinberg. After some days of work this is my first post on this platform and it seems to work.

    Why did I do it?

    Actually I have been programming Python for a while now. It is a great programming language and for me it was just fun to get to know more and more features of the language and available packages. I actually got into web-programming by beer... , yes beer... I'm not only a coder but also a homebrewer (I will write about my brewing experiences in this blog later on). To make my life as a homebrewer a bit easier I was working on a brewing controller, a Raspberry Pi controlling my mashing and brewing temperatures. I first came in touch with Flask when I was looking for a way to build a nice-looking GUI for the brewing controller. I immediately liked its simplicity and modularity. During the learning progress I came across Miguel Grinbergs "Flask Mega Tutorial" in which he points out the main features of Flask and some of its Addons by developing a microblogging platform. I liked it very much so I decided to buy his book and create my own blog basing on it. So now we're here.

    What will I write about?

    I want to write about my coding experiences especially by extending the functionality of this blog. Also I want to write about my brewing experiences and new beer creations. So I think there will be plenty to write about in the time to come. To keep the orientation one of my next objectives will be to add Tags to the posts.