Uploading Images in Django

Learning Objectives

Students will be able to:
Use a .env file for environment variables/secrets
Set up Amazon S3 to hold an app's uploaded images
Use an HTML <form> to send a file to the server
Upload a file to S3 from the server

Road Map

1. Intro
2. Ready the Starter Code
3. Using a .env File for "secrets"
4. Set up Amazon S3
5. Add a Photo Model
6. Add the add_photo URL
7. Add the add_photo View Function
8. Update the details.html Template
9. Test it Out!
10. Check it Out in Admin

Intro

Many applications we develop benefit from the ability of its users being able to upload and display images.

For example, an application that tracks a user's hikes would allow the user to upload photos of those hikes. Or better yet, our Cat Collector app can allow images to be uploaded for a cat!

This lesson will cover how to add this ability to your Django project should you choose to.

Ready the Starter Code

This lesson's starter code picks up from the many-to-many models lesson.

If your many-to-many code is working, you can continue working with that codebase, otherwise, sync your code:

• Move into your ~/code/catcollector folder: cd ~/code/catcollector

• Sync with the code-along repo's code: git fetch --all then git reset --hard origin/main

• Open the project in VS Code: code .

• Ensure the migrations have been run: python3 manage.py migrate

• Be sure that no other Django server is running, then start the server: python3 manage.py runserver

Using a .env File for "secrets"

caution

Don't ever put your Amazon AWS, or any other secret keys, in your source code!

If you do, and push your code to GitHub, it will be discovered within minutes and could result in a major financial liability!

To protect our "secrets", we can use a .env file locally like we did last unit.

Processing the .env file in our Django app will require the help from a separate package, django-environ.

Let's install it:

pip3 install django-environ

We can now use the environ module in catcollector/settings.py - add the following code anywhere in settings.py (below the other imports near the top of the file is fine):

# settings.py

import environ
environ.Env()
environ.Env.read_env()

Since we're processing the .env from the settings.py module, we will add our .env in the same folder:

touch catcollector/.env

We're now ready to add KEY=VALUE pairs like we did in the previous unit, for example:

SECRET_KEY=SOME_VALUE

To access the values in a Python app, we use the os.environ dict instead of Node's process.env:

from django.shortcuts import render, redirect
import os
...

def some_function(request):
    secret_key = os.environ['SECRET_KEY']

Note: We'll need to import os if not already imported.

Set up Amazon S3

AWS (Amazon Web Services) is a cloud platform offering a slew of web services such as compute power, database storage, content delivery and more.

You can see all of the services available by clicking here Amazon AWS.

The specific web service we are going to use to host our uploaded images is the Simple Storage Service, better known as Amazon S3.

Set up an Amazon AWS Account

Before we can use S3, or any Amazon Web Service, we'll need an Amazon AWS account.

Notice: Even though AWS has a "free tier" and we will do nothing in SEI that will cost money, AWS requires a credit card to open an account. If you don't wish to provide a credit card, you will not be able to complete this lesson and you won't be able to upload images using Amazon Web Services. Unfortunately, alternative services, such as Microsoft Azure, have the same credit card requirement.

Click the orange Sign in to the Console button, then click Create a new AWS account (or browse here):

Unfortunately, the sign up process is a bit lengthy...

Sign in to the AWS Console

After you have signed up, log in to the Console:

then sign in:

Create Access ID & Secret Keys

We need to do create an Access Key ID and a Secret Access Key to access S3 with.

Click the > All services drop-down:

Locate the "Security, Identity & Compliance" section and click IAM (Identity and Access Management):

Click Users in the sidebar:

Click the blue Add user button:

Enter a user name for the credentials being created (your web app's name if perfect), select Programmatic access, then click the Next Permissions button:

Now we need to create a group that will have S3 permissions that the user can be assigned to.

Click the Create group button:

Enter a Group name - django-s3-assets is fine.

Then search or scroll way down, select the AmazonS3FullAccess checkbox and click the blue Create group button:

On the following screen, ensure that the django-s3-assets group is selected, then click the blue Next: Tags button:

Now click the blue Next: Review button:

Then finally, click the blue Create user button:

Copy both access keys to a safe place, then add them to the catcollector/.env file:

AWS_ACCESS_KEY_ID=<paste your Access key ID>
AWS_SECRET_ACCESS_KEY=<paste your Secret access key>

Click the Close button (bottom-right), but don't close the browser - on to creating an S3 bucket...

Create an S3 Bucket for the catcollector App

Click the Services drop-down (top-left):

Click the S3 link under the "Storage" section.

Buckets are containers for stuff you store in S3.

Typically, you would want to create an S3 bucket for each web application you develop that needs to use S3.

Let's click the blue + Create bucket button to get started:

You'll have to enter a globally unique Bucket name, then select the nearest Region:

For the best performance, always be sure to select the nearest location to where your application will be hosted, e.g., Heroku. For this lesson, be sure to select the Region nearest you.

Make note of your region!

Click the Next button (bottom-right of popup) - TWICE to get to the following screen where you want to UNCHECK the Block all public access checkbox, then CHECK the I acknowledge... checkbox:

Click the blue Next button, then a new screen will appear where you will click the blue Create Bucket button!

The bucket has been created!

Let's add the bucket name to our .env:

AWS_ACCESS_KEY_ID=<paste your Access key ID>
AWS_SECRET_ACCESS_KEY=<paste your Secret access key>
S3_BUCKET=<your bucket name>

Next we need to make sure that web browsers will be able to download cat images when using catcollector.

To do so, we need to specify a "bucket policy" that enables read-only access to a bucket's objects.

Click on the bucket name:

Click the Permissions tab at the top:

Click on the square Bucket Policy button:

Then near the bottom click the Policy generator link:

A new tab will open in your browser.

Start entering the data as follows. BTW, that's a * in the "Principle" input.

This next one's a bit challenging. In the Amazon Resources Name (ARN) input enter this:

arn:aws:s3:::sei-catcollector/*

but substitute your bucket name for sei-catcollector.

Click the gold Add Statement button.

Once that is done, click the gold Generate Policy button that's in Step 3.

Copy the text inside the box, including the curly braces:

Now go back to the main tab and paste in that text:

Click the blue Save button (top-right).

You should see something like this at the top of the page:

Congrats! You now have an S3 bucket that, using the access keys, an application can upload files of any type to; that also allows any browser to download by using a known URL.

Now let's get on with the app!

Install Boto 3

The official Amazon AWS SDK (Software Development Kit) for Python is a library called Boto3.

Let's install it:

pip3 install boto3

Add a Photo Model

The Relationship

We're going to add a Photo Model that will hold the URL of a cat image in S3.

The relationship looks like this: Cat -----< Photo A Cat has many Photos -and- A Photo belongs to a Cat

Define the Photo Model

To the models.py file we go:

class Photo(models.Model):
    url = models.CharField(max_length=200)
    cat = models.ForeignKey(Cat, on_delete=models.CASCADE)

    def __str__(self):
        return f"Photo for cat_id: {self.cat_id} @{self.url}"

Nice and simple!

Reminder: If a Model "belongs to" another Model, it must have a foreign key. More than one "belongs to" relationship - means more than one foreign key.

What needs to be done now?

python3 manage.py makemigrations
python3 manage.py migrate

Add the add_photo URL

We need to add a new path() to the urlpatterns list that will match the request sent when the user submits a photo.

In urls.py:

urlpatterns = [
 ...
    path('cats/<int:pk>/delete/', views.CatDelete.as_view(), name='cats_delete'),
    path('cats/<int:cat_id>/add_feeding/', views.add_feeding, name='add_feeding'),
    # new path below
    path('cats/<int:cat_id>/add_photo/', views.add_photo, name='add_photo'),
]

Pretty much like the add_feeding route!

Notice once again, we're going to capture the cat's id using a URL parameter named cat_id.

The server currently shows an error because we've referenced an add_photo view function that doesn't exist. Let's take care of that next...

Code the add_photo View Function

Where do we define the view functions?

views.py

First, we need to import three more things:

• the boto3 library
• the Photo Model
• Python's uuid utility that will help us generate random strings

# views.py

from django.shortcuts import render, redirect
from django.views.generic.edit import CreateView, UpdateView, DeleteView
from .forms import FeedingForm
# new/updated imports below
import uuid
import boto3
from .models import Cat, Toy, Photo

Next, we're going to define a couple of variables we'll use in the view function...

Determine the correct AWS Service Endpoint

AWS has different endpoints (URLs) dependent upon the service (S3 in our case) and geographic region.

In regards to S3, the docs point to these endpoints for the regions nearest to our campuses:

• https://s3.us-east-2.amazonaws.com/ (Ohio)
• https://s3.us-west-1.amazonaws.com/ (N. California)

Pick an endpoint for the region you created the bucket in and let's add the URL to the .env file, for example:

AWS_ACCESS_KEY_ID=<paste your Access key ID>
AWS_SECRET_ACCESS_KEY=<paste your Secret access key>
S3_BUCKET=<your bucket name>
S3_BASE_URL=https://s3.us-west-1.amazonaws.com/

Note: Be sure to use your URL and include https:// in front of the endpoint and a trailing slash (/) at the end.

Each file uploaded to S3 must have a unique URL.

We'll be using S3_BASE_URL, S3_BUCKET and a randomly generated key to build this unique URL.

The unique URL will also be saved in the url attribute in each Photo instance.

Finally, this is where the magic happens, we'll review the code as we type it in:

# Add this import to access the env vars
import os

...

def add_photo(request, cat_id):
    # photo-file will be the "name" attribute on the <input type="file">
    photo_file = request.FILES.get('photo-file', None)
    if photo_file:
        s3 = boto3.client('s3')
        # need a unique "key" for S3 / needs image file extension too
        key = uuid.uuid4().hex[:6] + photo_file.name[photo_file.name.rfind('.'):]
        # just in case something goes wrong
        try:
            bucket = os.environ['S3_BUCKET']
            s3.upload_fileobj(photo_file, bucket, key)
            # build the full url string
            url = f"{os.environ['S3_BASE_URL']}{bucket}/{key}"
            # we can assign to cat_id or cat (if you have a cat object)
            Photo.objects.create(url=url, cat_id=cat_id)
        except Exception as e:
            print('An error occurred uploading file to S3')
            print(e)
    return redirect('detail', cat_id=cat_id)

Code like

key = uuid.uuid4().hex[:6] + photo_file.name[photo_file.name.rfind('.'):]

is a great line of code to examine in smaller pieces within the debugger if it seems overwhelming.

On to the UI...

Update the details.html Template

We're going to need to update the details.html template to:

• Display each image beneath the cat's detail "card" (left column).
• Show a "No Photos Uploaded" message if there are no images for the cat.
• Show a form below the images used to upload photos.

Display Cat Images

We're going to use Django's nifty for...empty template tags to iterate through each cat's photos like this:

...
    </div>
    <!-- Insert photo markup below this comment -->
    {% for photo in cat.photo_set.all %}
      <img class="responsive-img card-panel" src="{{photo.url}}">
    {% empty %}
      <div class="card-panel teal-text center-align">No Photos Uploaded</div>
    {% endfor %}
...

The for...empty template tags avoid having to wrap a for...in loop with an if...else like we did earlier to display a "No Toys" message.

Let's see how it looks:

Since we haven't uploaded any photos yet, we are seeing the No Photos Uploaded <div> as expected thanks to the {% empty %} tag.

Reminder - we don't actually invoke methods within Django template tags. For example, notice that there's no () in this line of code:

{% for photo in cat.photo_set.all %}

This is because Django templates automatically call an attribute if it's a function. This can be a problem if you ever need to actually call a function that takes arguments. For example, the following will not work:

{% if len(cat.photo_set.all) > 0 %}

Django templates provide filters for use in both {{ }} (variable) and {% %} (tags).

For example, to check the length, you can use the length filter like this:

{% if cat.photo_set.all|length > 0 %}

Filters can also be used to transform/format data, for example, here's how the capfirst filter can be used to capitalize the first letter of an outputted value:

{{ some_value|capfirst }}

or how the date filter can be used to format dates:

{{ some_date|date:"m/d/Y"}}

Form for Uploading Photos

Okay, let's code a <form> that we can use to upload files to the server:

{% for photo in cat.photo_set.all %}
    <img class="responsive-img card-panel" src="{{photo.url}}">
{% empty %}
    <div class="card-panel teal-text center-align">No Photos Uploaded</div>
{% endfor %}

<!-- new code below -->
<form action="{% url 'add_photo' cat.id %}" enctype="multipart/form-data" method="POST" class="card-panel">
    {% csrf_token %}
    <input type="file" name="photo-file">
    <br><br>
    <input type="submit" class="btn" value="Upload Photo">
</form>

When using HTML forms to upload files, it's important to add the enctype="multipart/form-data" attribute to the <form> tag.

Other than {% csrf_token %}, it's pretty much a generic form that is typically used to upload files to any web app.

Another refresh:

Looking good...

Test it Out

Try uploading your favorite cat pic.

Success!

Check it Out in Admin

Don't forget to register the new Photo model so that you can use the built-in Admin app to add/edit/delete model instances.

Update the admin.py file to look like this:

from django.contrib import admin
# Import your models here
from .models import Cat, Feeding, Photo

# Register you models here
admin.site.register(Cat)
admin.site.register(Feeding)
admin.site.register(Photo)

Then browse to localhost:8000/admin and click through to view the added photo.

You'll see something like this:

Note how the photo's url is formed.

Congrats on uploading files to Amazon S3

Lab

Incorporating the content of this lesson into your FinchCollector app is OPTIONAL.

Resources

Boto 3 Docs

Django Built-in Template Tags and Filters