The first part of this tutorial introduced some interesting aspects of Django 1.5. In this second part we are diving deeper into Django features but coding a little bit more, trying always to use the framework properly. Here is the index:
- Modifying the model.
- Improving posts administration.
- Adding commentaries.
- Tweaking the administrator to manage posts and commentaries.
- Adding a search function.
1. Modifying the model
Modifying the model is a normal task in big projects. In this case posts could have an owner to distinguish the author of an article or to ensure only the creator of a post can edit the post.
Let’s open models.py file inside posts folder and add a new field below publication_date:
owner = models.ForeignKey(auth.models.User)
Note how User is no more than a model provided by the auth application of the Django framework. With models.foreignKey we are adding a one-to-many relationship between Post and User entities. The model containing the foreignKey is the part of the relationship with only one of the other entities. This creates automatically a virtual property in the other part of the relationship but let’s return on this in a while.
We could verbalize the relationship as:
A Post is related with only one User
As we know the nature of this relationship we can state:
A Post belongs to only one User
We are changing the model by adding a new field so the database should be updated. To do this automatically we use south. First type:
$ ./manage.py schemamigration posts --auto
The parameter auto makes south to try to discover automatically what happen to the model. In this case we are adding a field and south realized about this. The field can not be null but we are not providing any default value so the migration procedure is asking the developer what to do now. Choose 2nd option to specify a value manually and press enter. Now you are in a python terminal and you must provide a literal. Introduce number 1 (User 1 is the administrator user you create in the other tutorial).
Now its time to apply the migration:
$ ./manage.py migrate posts
Run the server and go to the administrator URL to check how all the posts now contain a new field called owner. The new form field includes a green plus icon which allows the administrator to add a new user to the database without moving outside the post editing view. Once created, it is bound to the current post. Try adding some users to the database. Use the admin panel to edit their details and fulfill their names and last names and create new posts with those owners.
2. Improving post administration
We can improve how to manage the posts in the administrator associating an administration model at the time we register the model. To do this, edit the admin.py file of posts application and leave it as follows:
from posts.models import Post from django.contrib import admin class PostAdmin(admin.ModelAdmin): list_display = ('title', 'excerpt', 'publication_date', 'owner') list_filter = ['publication_date', 'owner'] date_hierarchy = 'publication_date' search_fields = ['title', 'content', 'owner__username', 'owner__first_name', 'owner__last_name'] prepopulated_fields = { 'machine_name' : ('title', ) } admin.site.register(Post, PostAdmin)
The new class PostAdmin is an administrator model which controls how the list of all posts and the view for creating / editing a post should look. Note how the admin model is not bound (despite of the name convention) to any actual model until line 15 where we pass the model and the associated admin model. By overriding some fields we can change the information to show in the admin website or the behavior for the administration forms. Some of these fields are:
- list_display establishes which fields will be shown in the list of posts.
- list_filter adds a date filter to the right of this list.
- date_hierarchy enables a navigable date hierarchy on the specified date. It is placed just over the list, below the search box.
- search_fields points to the list of fields where the text introduced in the search box will be looked for.
- prepopulated_fileds is used to populate automatically some fields in function to other fields.
Observe line 11. The ‘__’ notation is used to reach fields from other models. In this case we want to search into the user name, first name and last name of the owner of the post. Thus, owner__first_name is equivalent to owner.first_name and owner__last_name to owner.last_name.
A list of all options you can specify in an administration model is provided in the Django documentation.
Run the server an try the administration panel for Posts. Observe how in the list of all posts you can click on the column headers to set the sorting criteria. Sorting is limited to static fields, no order can be specified on calculated fields like excerpt.
3. Adding commentaries
The next version of Django has marked comment framework as deprecated so let’s anticipate this by implementing a custom one ourselves. We want commentaries for our posts. A commentary has a content, an author, a publication date and it’s related with the post which the commentary belongs to.
So, go inside the posts application folder and open models.py. Add a new model as follows:
class Commentary(models.Model): post = models.ForeignKey(Post) content = models.TextField() publication_date = models.DateTimeField(auto_now_add=True) author = models.CharField(max_length=50, default=u'The invisible man') def __unicode__(self): return self.owner + u'@' + unicode(self.post) class Meta: verbose_name_plural = u'commentaries' ordering = [u'-publication_date']
The field post is a foreign key setting one-to-many relationship between Commentary and Post entities:
A Commentary belongs to only one Post
As you see again, the foreign key is in the part of the relationship which has only one of the other entities. Note you have not modified Post model but, as a consequence of this foreign key, a virtual property is added to the Post class to represent the set of Commentary entities the Post is related to. This property is named commentary_set.
We have to update the database before continuing so, in a terminal, run:
$ ./manage.py schemamigration posts --auto $ ./manage.py migrate posts
A form to add commentaries
Forms are set of fields to be fulfilled with some constrains. We need to build a form with almost the same fields (some of them, like publication_date are filled automatically) so we are using a base class ModelForm able to inspect a model and to create the proper fields for the form.
Inside posts application folder create a new file named forms.py and add the following content:
# -*- encoding: utf-8 -*- from django import forms from posts.models import Commentary class AddCommentaryForm(forms.ModelForm): class Meta: model = Commentary fields = ('owner', 'content')
Overriding the fields property we are indicating explicitly what fields we want to provide to fulfill. The order is important because it is the order in which the fields will be shown. We don’t want the user to complete publication_date nor post fields. The first one is set automatically meanwhile the second should be provided automatically because it is the post we are commenting on.
The post view
We are going to create a view to display a solely post and its commentaries. Open views.py and import the class TemplateView as well as AddCommentaryForm we created before.
from django.views.generic import ListView, TemplateView from posts.forms import AddCommentaryForm
Now add this code at the end of the file:
class PostDetails(TemplateView): template_name = 'postdetails.html' def post(self, request, *args, **kwargs): return self.get(request, *args, **kwargs) # Overriding def get_context_data(self, **kwargs): context = super(PostDetails, self).get_context_data(**kwargs) post = self.get_post(kwargs['slug']) form = self.get_form(post) context.update({'form':form, 'post':post}) return context # Helper def get_post(self, slug): return Post.objects.get(pk=slug) # Helper def get_form(self, post): if self.request.method == 'POST': form = AddCommentaryForm(self.request.POST) if form.is_valid(): commentary = form.save(commit=False) post.commentary_set.add(commentary) else: return form return AddCommentaryForm()
The TemplateView is a fully customizable view with the only intention to provide a template as answer. The template can be set by overriding the template_name property. In this case, we provide the template postdetails.html which we will create in the next section.
By providing HTTP verbs we enable the view to handle different kind of requests. By default, only GET requests are allowed. The default behavior is to call get_context_data() method and send the result to the template as the context object we talk in the first part of the tutorial.
As we need the view to handle POST requests, we provide the method post() in line 4. The method is quite simple and just delegates on get() method.
By default get_context_data() returns a dictionary with the key params which contains the parameters supplied on the query string of the request. We need to include the post the user is visiting and the commentary form so we override get_context_data() in line 8. First let the default behavior takes place (line 9) and then get the post the user is visiting from the URL (line 11) and the form for commentaries (line 12). Finally updates the context object with the form and the post and return the new context.
The method get_post() accepts the slug provided in the URL. In the next section we will create a URL schema to identify which part of the URL is the slug. Once we know the slug we use it to retrieve the only object whose public key (pk in line 20) is the slug. The get() method of a collection tries to retrieve a unique entity. If there is some ambiguity and more than one fulfill the constrains specified as parameters, an exception is raised.
The method get_form() accepts a post to use in case of handling a POST request for a commentary. We need to distinguish two scenarios:
- We are visiting a post (GET request)
- We are sending a commentary (POST request)
In any case (except if the form is invalid) we will return an empty form (line 32) ready to receive another commentary but if we are handling a valid POST request, we must add the commentary to the post. Fore so, in case we are attending a POST request (line 24), we try to construct a new AddCommentaryForm populated by the contents of the POST payload (line 25). If this form is valid (line 26) save the form to produce a commentary object (line 27). Parameter commit set to false is necessary as we are not providing the post field through the form so we cannot write the object to the database. Saving the form with commit set to false produce the Commentary object but avoid writing it into the database.
In the other hand, if the form is invalid, the validation process put errors inside the form object so we return the invalid form.
As I commented before, each time a one-to-many relationship is created, a virtual property named from the relationship is created in the other part of the relationship. In this case the property is part of the Post class and it’s called commentary_set. In line 28 we access this set and add it the new commentary. In this way we modify the relationship from the other end: instead of setting the post in the commentary, we add the commentary to the post.
URL schema for posts
Now we have the view, lets think about the URL schema. The URL schema is referred as the set of patters you use to specify how to build the URLs to navigate your site. It does not refer to the URLs but how we build the URLs. The schema we present here is simple and follows some REST conventions:
- /posts/ – refer to the collection of posts
- /posts/<machine_name_of_the_post>/ – refer to the post with the provided machine name
Django uses named groups and regular expressions to recognize parts of an URL. So the regular expressions for former URLs are:
r'^/posts/$' r'^/posts/(?P<slug>[a-zA-Z0-9_-]+)/$
The first one says something like:
A string that start with ‘/’, followed by ‘posts’ and ending with ‘/’
The second one means:
A string starting with ‘/’, followed by ‘posts/’, with a sub-string of at least 1 character of letters, numbers, underscores or hyphens named ‘slug’ and ending with /
We could add these two patterns to alacola/urls.py in the same way we did in the first part of the tutorial but let’s split the URL patterns in such a way the part related with posts goes inside posts application. We ignore ‘posts/’ part of the URL because it could be named in several other ways and focus on the behavior after the slash:
- No machine_name is provided, display the list of posts
- A machine_name is provided, display the post with that machine_name
So go into the posts application folder and add a new file named urls.py. Leave it like:
# -*- encoding: utf-8 -*- from posts.views import PostList, PostDetails from django.conf.urls import patterns, include, url urlpatterns = patterns('', url(r'^$', PostList.as_view(), name='postlist'), url(r'^(?P<slug>[a-zA-Z0-9_-]+)/$', PostDetails.as_view(), name='postdetails'), )
Before continuing, pay attention to name parameters. They allow to provide a name to the URL in order to refer the pattern when needed. Now go to djangoblog/urls.py and leave it like:
from django.conf.urls import patterns, include, url from django.views.generic.base import RedirectView # Uncomment the next two lines to enable the admin: from django.contrib import admin admin.autodiscover() urlpatterns = patterns('', # Examples: # url(r'^$', 'djangoblog.views.home', name='home'), # url(r'^djangoblog/', include('djangoblog.foo.urls')), # Uncomment the admin/doc line below to enable admin documentation: url(r'^admin/doc/', include('django.contrib.admindocs.urls')), # Uncomment the next line to enable the admin: url(r'^admin/', include(admin.site.urls)), url(r'^$', RedirectView.as_view(url='/posts/')), url(r'^posts/', include('posts.urls')), )
We have changed some things since the last edition. Observe line 19. When accesing root URL, instead of using a custom view, we use a Django-provided generic view RedirectView to cause a redirection to /posts/ URL. When accessing an URL starting with /posts/ (line 20) we say Django to use the patterns inside posts application by indicating its urls module.
Note how in line 20, the pattern does not say «ending with /» as we need to cover both scenarios: when a machine name is provided and when it’s not. When using include() function, not the complete URL is passed to the included patterns: the already matching part of the URL is stripped out first.
So when /posts/an-interesting-article/ is requested, the /posts/ part matches the pattern in line 20 and the string an-interesting-article/ is passed to the patterns in posts/urls.py matching the pattern in line 7. This sub-string is then grouped following the regular expression group rules so an-interesting-article is given the name slug. Finally the dispatcher creates a PostDetails object and pass a HttpRequest object and a dictionary with the names of the groups as keys and the respective contents of each group as values.
If you run the server, try going to the root URL and you will be redirected to the post list. If you try to access a post knowing its machine_name you will get a TemplateDoesNotExists error because there is no template yet.
The post template
At last, it’s time to write the template showing post details. Let’s first tweak the post list to include how many commentaries has a post. Edit postlist.html inside templates folder in the djangoblog application folder. Go and replace the line saying:
<!-- Here will be commentaries -->
With this new code:
<p> <a href="{{ post.get_absolute_url }}#comments"> <span class="badge badge-info">{{ post.commentary_set.count }}</span> commentar{{ post.commentary_set.count|pluralize:"y,ies" }} </a> </p>
Now replace the post header by:
<h2><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h2>
The attribute get_absolute_url of any Django model is a convenient way to get the unique URI for the object. We need to add it to the Post model so open models.py inside posts application and add the following method to Post class:
def get_absolute_url(self): from django.core.urlresolvers import reverse return reverse('postdetails',kwargs={ 'slug':self.machine_name })
Do you remember the name parameter of URL pattern. Let’s refresh your memory:
url(r'^(?P<slug>[a-zA-Z0-9_-]+)/$', PostDetails.as_view(), name='postdetails'),
So with reverse() function you can provide a URL name and parameters for the named groups to rebuild the URL. This is what we are just doing here. Now revisit the post list and check how headers and commentaries redirect to the proper URL.
Go to templates directory inside djangoblog application, create a new file named postdetails.html and add this content:
{% extends "base.html" %} {% block title %} {{ post.title }} {% endblock %} {% block content %} <article class="post"> <h1>{{ post.title }}</h1> <section> {{ post.content }} </section> <aside> <p class="label label-info"> Published on <time>{{ post.publication_date|date:"r" }}</time> </p> <p> <span class="badge badge-info">{{ post.commentary_set.count }}</span> commentar{{ post.commentary_set.count|pluralize:"y,ies" }} </p> </aside> <h2 id="comments">Leave a commentary</h2> <section> <form action="{% url "postdetails" slug=post.machine_name %}" method="POST"> {% csrf_token %} {{ form.as_p }} <input type="submit" value="Send" /> </form> {% for commentary in post.commentary_set.all %} <article class="well"> <p>{{ commentary.owner }} said on <time>{{ commentary.publication_date|date:"r" }}</time>:</p> <blockquote>{{ commentary.content }}</blockquote> </article> {% if not forloop.last %} <hr/> {% endif %} {% empty %} <p class="label label-info">No comments at the moment.</p> {% endfor %} </section> </article> {% endblock %}
New tags and filters are coming! To include a form in a template the simplest way, use form.as_p attribute. You can use other approaches to include the form in a table or fully control how the fields must to be displayed. The point is that you need to add the form tag and the submit button manually. Do not forget to provide the action URL and method!
Pay attention to line 23. You can find the first new tag: url. The tag is very similar to reverse() function, it takes the name of an URL pattern and the content for the named groups and rebuild the absolute URL. This way we indicate we want to POST to the post’s URI. We could have used post.get_absolute_url but I want to show the url tag because it is a very useful tag for when we are not dealing with objects URIs.
The second new tag is csrf_token. It is part of the CSRF protection and must be included in every POST form. It acts like a signature in such a way Django can recognize the post has been sent from the Django application and it has not been forged by an attacker.
Tag for is an old friend but now we use the empty (line 36) tag inside to specify the behavior when the collection is empty.
In the other hand, if tag is new. It is used to test conditions and take decisions in the same way you use if in the code. You can use several boolean operators. Inside a loop you can access loop variables such as last or first that are true if it is the last or first iterations.
Now you have postdetails.html so you can run your server and access some post. Try to add a new commentary and see what happen.
4. Managing commentaries
We are adding Commentary administration by registering it into the Admin site so open admin.py, ensure your are importing Commentary model and add this controller:
from posts.models import Commentary class CommentaryAdmin(admin.ModelAdmin): list_display = ('owner', 'post', 'publication_date') list_filter = ['publication_date', 'owner'] search_fields = ['owner', 'content', 'post__title'] admin.site.register(Commentary, CommentaryAdmin)
Try to run the server and add some commentaries and you’ll realize adding commentaries is a little bit annoying. It could be better if we could modify commentaries at the same time we manage posts. This is possible by adding inline administrator models. Inline administrator models allow editing models inside a parent model. So, add the CommentaryInline model to admin.py and modify PostAdmin to override inlines property:
class CommentaryInline(admin.StackedInline): model = Commentary class PostAdmin(admin.ModelAdmin): list_display = ('title', 'excerpt', 'publication_date', 'owner') list_filter = ['publication_date', 'owner'] date_hierarchy = 'publication_date' search_fields = ['title', 'content', 'owner__username', 'owner__first_name', 'owner__last_name'] prepopulated_fields = { 'machine_name' : ('title', ) } inlines = [CommentaryInline]
Now got to admin again, go to an existing post and test the new way of adding, editing or removing commentaries.
5. Enabling search
To finish, we are going to reinforce these concepts with a new form but at the same time I’m going to introduce you to context processors. You will add a search box. The search form must be available to all pages in the site so it would be convenient to have some mechanism to add the form to any context sent to a template automatically instead of modify every single view in our project.
Context processors are the mechanisms we are looking for. They are functions that receive a request and return a dictionary. This dictionary updates the context being sent to a template. Unfortunately the settings containing context processors is not in settings.py by default so you need to add it manually. Luckily, the Django documentation show the default value so open settings.py and add this to the end of the file:
TEMPLATE_CONTEXT_PROCESSORS = ( "django.contrib.auth.context_processors.auth", "django.core.context_processors.debug", "django.core.context_processors.i18n", "django.core.context_processors.media", "django.core.context_processors.static", "django.core.context_processors.tz", "django.contrib.messages.context_processors.messages", )
Overview
Think about the complete functionality: we are going to provide a search form for our whole site in the title bar. When clicking on search we are looking for posts containing the string into the search box and display them into a new page. Thus, we need:
- An URL schema for search – let it be /search/?query=<search string>
- A template to display the results
- A view where to perform the actual search and get the results
- A place in the blog to display the search box
- A context processor to add the search form to all views
- A form to be displayed as the search box
In the future, we could look for other entities rather than only posts so we will add this functionality to djangoblog application instead of posts. From bottom to top…
The search box form
This time the search box is not based on any model so we are going to create a basic form inheriting directly from Form. Add a new forms.py file inside djangoblog application and add this content:
# -*- encoding: utf-8 -*- from django import forms class SearchForm(forms.Form): query = forms.CharField(min_length=3, required=False)
Here we can see how a plain form is defined. It has no much logic, we simply put the fields as attributes a set some constrains in field constructors. In this case a simple char field with a minimum length of three characters.
The context processor
Now we need to insert the form in any context so add a context_processors.py file to djangoblog application and leave it like:
# -*- encoding: utf-8 -*- from djangoblog.forms import SearchForm def searchform(request): if request.method == 'GET': form = SearchForm(request.GET) else: form = SearchForm() return { 'searchform' : form }
Now add the context processor to the tuple of context processors you add to settings.py before. The new item is:
"djangoblog.context_processors.searchform",
So with this context processor we are adding an empty form if posting or a filled one if getting.
The search box template
This is pretty simple, we are going to modify base.html inside templates folder in djangoblog application to include the search box form. The only difference is now we are not using as_p method but controlling how to display each field. Replace the line with the title of the blog:
<a class="brand" href="#">My Django blog</a>
with the following alternative content:
<a class="brand pull-left" href="/">My Django blog</a> <form action="{% url "search" %}" method="GET" class="navbar-search pull-right"> <input type="search" name="{{ searchform.query.name }}" value="{{ searchform.query.value }}" class="search-query" placeholder="Search"/> </form>
You can see how I provide the widget to display as search box manually only consulting the searchform property of the context when getting the name and the value. This way you have full control over the widget and its properties. There is no send button, pressing enter suffices.
The results view
More logic is needed to ask the database for those posts containing the query string. Add a views.py file in djangoblog application and add the following code:
# -*- encoding: utf-8 -*- from django.views.generic import ListView from djangoblog.forms import SearchForm from posts.models import Post class SearchResults(ListView): template_name = "searchresults.html" # Override def get_queryset(self): if self.request.method == 'GET': form = SearchForm(self.request.GET) if form.is_valid(): query = form.cleaned_data['query'] results = Post.objects.filter(content__icontains=query) return results return Post.objects.none()
As you can see this view is another ListView but this time we have overridden (line 5) the get_queryset() method to display only those posts which content contains the query ignoring the letter case (line 10). If form is invalid (line 8) or we are not processing a GET request (line 6), return the empty collection (line 13).
This code deserves a deeper explanation of what are we doing. Before querying the database, the view checks if it’s handling a GET (line 6) request. If so, it tries to build a new SearchForm (line 7).
Now look at line 9, we are assigning query from cleaned_data dictionary of the form. When a form is created, the data in the query string or POST content is converted to objects of proper type following the form definition. After validation, the valid values are stored in the cleaned_data member.
Let’s explain more about querying a collection. In Django, accessing members of a model is done by performing operations over Model.objects member. We’ve already seen the method get() to access an object but if we want to get more than one instance then we have to use filter() instead. Other methods are available as well such as all() or exclude(). These methods return queryset instances and are chainables. Queryset are not tuples nor lists, they act like cursors inside the database so the memory consumption is low.
You can pass named parameters to some methods such as filter(). Normally, parameters have the same name as some model field over which the method should act. A suffix starting with __ (double underscore) indicates a test and the value assigned to the parameter represents the value to test. If no test is provided, it is an equality test.
Do you remember when using get()?
Post.objects.get(pk=slug)
This means:
Gets the unique post whose public key is equal to slug
Now using filter():
Post.objects.filter(content__icontains=query)
Which means:
Only those posts whose content contains (ignore case) query
The results view
We are almost done. We need a template to show the results so open templates folder and add a file named searchresults.html to the directory. Add these lines:
# -*- encoding: utf-8 -*- from django import forms class SearchForm(forms.Form): query = forms.CharField(min_length=3, required=False) {% extends "base.html" %} {% block title %}Search results{% endblock %} {% block content %} <h1>Search results:</h1> {% for post in object_list %} <h2><a href="{{ post.get_absolute_url }}">post.title</a></h2> {% if not forloop.last %} <hr/> {% endif %} {% empty %} <p class="label label-warning">No results for your query.</p> {% endfor %} {% endblock %}
The URL schema
The last remaining thing to do is to bind the URL pattern with the view. Edit urls.py from djangoblog and add another entry with the following content:
url(r'^search/', SearchResults.as_view(), name="search"),
We are done. Start the server and try to find some text you know it is in some post and a random string to check what happen when there is no results.
Search improvement using Q objects
Before finishing, let’s improve a little bit the search mechanism. It should be great if, instead of looking for the whole string inside the post content, we looked for each word returning those posts containing all the words in the title or in the content.
Modify the SearchResults.get_queryet() method in views.py inside djangoblog application to make it look like:
def get_queryset(self): from django.db.models import Q if self.request.method == 'GET': form = SearchForm(self.request.GET) if form.is_valid(): query = form.cleaned_data['query'] words = query.split(' ') qobjects = [Q(content__icontains=w) | Q(title__icontains=w) for w in words] condition = reduce(lambda x,y: x & y, qobjects) results = Post.objects.filter(condition) return results return Post.objects.none()
In few words, Q objects are used to build WHERE clauses in SQL. Two Q objects can be combined by using | or & in Python in the same way you use OR or AND in SQL.
The explanation for the following code is, starting from line 5, as follows:
- Get the query from the form.
- Get the words in a list by breaking the string in the white spaces.
- Make a list of Q objects to look for each word in content or title by combining using OR two separate Q objects.
- Reduce the former list by combining all the Q conditions with AND
- Filter by that condition
- Return the results
And now yes, we are done!
Conclusions
The long walk to here has covered a lot of Django concepts including advanced queries using Q objects. Even, there is lot of features you can not miss so I encourage you to read Django documentation. As in Python, before doing things by yourself, look for the solution first. Django is a very mature framework and it is probably a solution already exists or there is a partial solution you can extend.
This tutorial does not cover a very important topic when working with Django: deploying Django. But there are lots of resources out there. I recommend you the official documentation again and the site http://www.deploydjango.com/.
It has been a long way to this point. At last, the Django 1.5 in a nutshell series is done… or not? I’m thinking in adding a bonus chapter to these tutorials in order to cover REST APIs and AJAX. I will be partly motivated by the Python mentoring program I’m involved in. Let’s see.
Finally, I’m very happy with my first incursions to English posts. Now its time to continuing translations. I’m thinking do it in visiting order. As I realized with these posts, it is very difficult just translating posts, more if I’ve learned new things since the time I wrote them the first time. Hence, I think the new coming posts will be better.
See you in the next translation! Please send feedback to commentaries!
Thanks a lot.