JIYIK CN >

Python is the most popular programming language for data science, and MongoDB, with its flexible and dynamic schema, is a great combination for creating modern web applications, JSON APIs, and data processors, to name a few.

MongoDB also includes a native Python driver and a team of engineers dedicated to ensuring MongoDB and Python work seamlessly together.

Python provides extensive support for common data manipulation and processing operations. For example, Python's native dictionary and list data structures are second only to JavaScript when it comes to processing JSON documents, making them ideal for use with BSON.

PyMongo, the official Python MongoDB driver library, is equally simple and provides an intuitive API for accessing databases, collections, and documents.

Objects retrieved from MongoDB using PyMongo are compatible with dictionaries and lists, allowing for simple manipulation, iteration, and printing.

Storing data in MongoDB

MongoDB stores data in JSON-like documents:

# Mongodb document (JSON-style)
document_1 = {
"_id" : "BF00001CFOOD",
"item_name" : "Bread",
"quantity" : 2,
"ingredients" : "all-purpose flour"
}

A Python dictionary looks like this:

# python dictionary
dict_1 = {
"item_name" : "blender",
"max_discount" : "10%",
"batch_number" : "RR450020FRG",
"price" : 440
}

Python prerequisites and installation

Download and install Python on your machine. Confirm your installation is correct by typing python in a command line window.

You should get the following:

Python 3.9.1 (tags/v3.9.1:9cf6752, Feb  5 2021, 10:34:40) [MSC v.1927 64 bit (AMD64)] on win32
>>>

If you are new to Python, you can follow along with the Python MongoDB examples in this lesson.

Connect to MongoDB database locally using Python

PyMongo provides a set of libraries for working with MongoDB in Python. To get PyMongo up and running, open a command prompt and type the following:

python -m pip install pymongo

For this Python MongoDB tutorial, you will use the MongoDB SRV URI. So let's install dnspython:

python -m pip install dnspython

Now, you can use PyMongo as the Python MongoDB library in our code using the import statement. But first, let's create a MongoDB database in Python.

Therefore, the first step to connect Python is MongoDB cluster setup.

Next, write the PyMongo code into a pymongotestinsert.py file in any subdirectory. Any simple text editor, such as Textpad/Notepad, will suffice.

Add the following line to your MongoDB client:

def get_database():
    from pymongo import MongoClient
    import pymongo

    # Provide the mongodb url to connect python to mongodb using pymongo
    CONNECTION_STRING = "mongodb+srv://<username>:<password>@<cluster-name>.mongodb.net/myFirstDatabase"

    # Create a connection using MongoClient. You can import MongoClient or use pymongo.MongoClient
    from pymongo import MongoClient
    client = MongoClient(CONNECTION_STRING)

    # Create the database for your example
    return client['user_shopping_list']

    # This is added so that files can reuse the function get_database()
    if __name__ == "__main__":

    # Get database
    dbname = get_database()

To get a MongoDB database connection, create a Mongo client using CONNECTION_STRING. First change the cluster name, username, and password.

In this Python MongoDB course, you will make a shopping list and add some products. You create a database called UserShoppingList for this purpose.

MongoDB, on the other hand, doesn't build a database until you add collections and documents to it. So, next, let's make a collection.

Creating Sets in Python

To create a collection, pass the collection name to the database. Make sure the indentation is correct when you copy the code into a .py file.

collection_name = dbname["user_1_items"]

This will create a collection called user_1_items in the user_shopping_list database.

Inserting documents in Python

Use PyMongo insert_many()methods to insert multiple documents at once.

item1 = {
"_id" : "U1IT00001",
"item_name" : "Blender",
"max_discount" : "10%",
"batch_number" : "RR450020FRG",
"price" : 440,
"category" : "kitchen appliance"
}

item2 = {
"_id" : "U1IT00002",
"item_name" : "Egg",
"category" : "food",
"quantity" : 12,
"price" : 50,
"item_description" : "brown country eggs"
}
collection_name.insert_many([item1, item2])

Insert the third document without mentioning the _id field. This time, you will include a date data type field.

Use the Python dateutil module to add dates in PyMongo. Because ISODate is a Mongo shell function, it does not work with Python.

python -m pip install python-dateutilis the command to install the package. Then, in pymongo test.py, add the following:

from dateutil import parser
expiry_date = '2021-07-13T00:00:00.000Z'
expiry = parser.parse(expiry_date)
item3 = {
"item_name" : "Bread",
"quantity" : 2,
"ingredients" : "all-purpose flour",
"expiry_date" : expiry
}
collection_name.insert_one(item3)

The insert one()method is used to insert a single document.

First, use the command line to navigate to the location where you saved pymongo test insert.py. Then, run the file using the Python pymongo test insert.py command.

Querying in Python

You can use find()to view all documents together. To do this, you will create a separate file, pymongo_test_query.py:

# Get the database using the method you defined in the pymongo_test_insert file
from pymongo_test_insert import get_database
dbname = get_database()

# Create a new collection
collection_name = dbname["user_1_items"]

item_details = collection_name.find()
for item in item_details:
    # This does not give a readable output
    print(item)

Use the command line to navigate to pymongo test query.pythe folder where you saved . You can use the python pymongo test query.py command and run the program.

The data is visible but the formatting is not ideal. So here are the item names and their categories to be printed:

print(item['item_name'], item['category'])

Even though MongoDB received all the data, you will get a python KeyError on the third document. Handle missing data issues in Python DataFrames using pandas.

DataFrame is a two-dimensional data structure used for data processing. For example, Pymongo's discover()method returns a dictionary object, which can be converted to a data frame with just one line of code.

Install the pandas library as:

python -m pip install pandas

Replace the for loop with the following code to handle KeyError in one step:

from pandas import DataFrame
# convert the dictionary objects to a data frame
items_dfs = DataFrame(item_details)
print(items_dfs)

Remember to print(item['item name'], item['category'])comment out . For missing values, NaN and NaT are used to replace errors.

Indexes in Python and MongoDB

The number of documents and collections in real-world databases continues to grow. In an extensive collection, searching for a specific document—for example, a record containing all-purpose flour as an ingredient—can take a long time.

Indexes make database searches faster, more efficient, and less expensive—for example, sorting, counting, matching, and so on.

At the collection level, MongoDB defines indexes. It adds new documents to our collection, making the indexes more logical.

Using the insert many()method, you can insert multiple documents at the same time.