Using only the DB part of Django

Does somebody know how "modular" is Django? Can I use just the ORM part, to get classes that map to DB tables and know how to read/write from these tables?

If not, what would you recommend as "the Python equivalent of Hibernate"?

14 Answers
14

The short answer is: no, you can't use the Django ORM separately from Django.

The long answer is: yes, you can if you are willing to load large parts of Django along with it. For example, the database connection that is used by Django is opened when a request to Django occurs. This happens when a signal is sent so you could ostensibly send this signal to open the connection without using the specific request mechanism. Also, you'd need to setup the various applications and settings for the Django project.

Ultimately, it probably isn't worth your time. SQL Alchemy is a relatively well known Python ORM, which is actually more powerful than Django's anyway since it supports multiple database connections and connection pooling and other good stuff.

Edit: in response to James' criticism elsewhere, I will clarify what I described in my original post. While it is gratifying that a major Django contributor has called me out, I still think I'm right :)

First off, consider what needs to be done to use Django's ORM separate from any other part. You use one of the methods described by James for doing a basic setup of Django. But a number of these methods don't allow for using the syncdb command, which is required to create the tables for your models. A settings.py file is needed for this, with variables not just for DATABASE_*, but also INSTALLED_APPLICATIONS with the correct paths to all models.py files.

syncdb

DATABASE_*

INSTALLED_APPLICATIONS

It is possible to roll your own solution to use syncdb without a settings.py, but it requires some advanced knowledge of Django. Of course, you don't need to use syncdb; the tables can be created independently of the models. But it is an aspect of the ORM that is not available unless you put some effort into setup.

syncdb

syncdb

Secondly, consider how you would create your queries to the DB with the standard Model.objects.filter() call. If this is done as part of a view, it's very simple: construct the QuerySet and view the instances. For example:

Model.objects.filter()

QuerySet

tag_query = Tag.objects.filter( name='stackoverflow' )
if( tag_query.count() > 0 ):
tag = tag_query[0]
tag.name = 'stackoverflowed'
tag.save()


Nice, simple and clean. Now, without the crutch of Django's request/response chaining system, you need to initialise the database connection, make the query, then close the connection. So the above example becomes:

from django.db import reset_queries, close_connection, _rollback_on_exception
reset_queries()
try:
tag_query = Tag.objects.filter( name='stackoverflow' )
if( tag_query.count() > 0 ):
tag = tag_query[0]
tag.name = 'stackoverflowed'
tag.save()
except:
_rollback_on_exception()
finally:
close_connection()


The database connection management can also be done via Django signals. All of the above is defined in django/db/init.py. Other ORMs also have this sort of connection management, but you don't need to dig into their source to find out how to do it. SQL Alchemy's connection management system is documented in the tutorials and elsewhere.

Finally, you need to keep in mind that the database connection object is local to the current thread at all times, which may or may not limit you depending on your requirements. If your application is not stateless, like Django, but persistent, you may hit threading issues.

In conclusion, it is a matter of opinion. In my opinion, both the limitations of, and the setup required for, Django's ORM separate from the framework is too much of a liability. There are perfectly viable dedicated ORM solutions available elsewhere that are designed for library usage. Django's is not.

Don't think that all of the above shows I dislike Django and all it's workings, I really do like Django a lot! But I'm realistic about what it's capabilities are and being an ORM library is not one of them.

P.S. Multiple database connection support is being worked on. But it's not there now.

If you like Django's ORM, it's perfectly simple to use it "standalone"; I've written up several techniques for using parts of Django outside of a web context, and you're free to use any of them (or roll your own).

Shane above seems to be a bit misinformed on this and a few other points -- for example, Django can do multiple different databases, it just doesn't default to that (you need to do a custom manager on the models which use something other than the "main" DB, something that's not too hard and there are recipes floating around for it). It's true that Django itself doesn't do connection management/connection pooling, but personally I've always used external tools for that anyway (e.g., pgpool, which rocks harder than anything built in to an ORM ever could).

pgpool

I'd suggest spending some time reading up and possibly trying a few likely Google searches (e.g., the post I linked you to comes up as the top result for "standalone Django script") to get a feel for what will actually best suit your needs and tastes -- it may be Django's ORM isn't right for you, and you shouldn't use it if it isn't, but unfortunately there's a lot of misinformation out there which muddies the waters.

Editing to respond to Shane:

Again, you seem to be misinformed: SQLAlchemy needs to be configured (i.e., told what DB to use, how to connect, etc.) before you can run queries with it, so how is the fact that Django needs similar configuration (accomplished via your choice of methods -- you do not need to have a full Django settings file) any disadvantage?

As for multiple DB support, you seem to be confused: the support is there at a low level. The query object -- not QuerySet, but the underlying Query object it will execute knows what DB it's connecting to, and accepts a DB connection as one of its initialization arguments. Telling one model to use one DB and another model to use another is as simple as setting up one method on a manager which passes the right connection info down into the Query. True, there's no higher-level API for this, but that's not the same as "no support" and not the same as "requires custom code" (unless you'd argue that configuring multiple DBs explicitly in SQLAlchemy, required if you want multiple DBs, is also "custom code").

QuerySet

Query

Query

As for whether you end up indirectly using things that aren't in django.db, well, so what? The fact that django.db imports bits of, say, django.utils because there are data structures and other bits of code which are useful for more than just an ORM is fine as far as I'm personally concerned; one might as well complain if something has external dependencies or makes use of standard Python libraries instead of being 100% self-contained.

django.db

django.db

django.utils

sqlalchemy

You can certainly use various parts of Django in a stand-alone fashion. It is after-all just a collection of Python modules, which you can import to any other code you would like to use them.

I'd also recommend looking at SQL Alchemy if you are only after the ORM side of things.

(I'm reporting my solution because my question said to be a duplicate)

Ah ok I figured it out and will post the solutions for anyone attempting to do the same thing.

This solution assumes that you want to create new models.

First create a new folder to store your files. We'll call it "standAlone". Within "standAlone", create the following files:

__init__.py
myScript.py
settings.py


Obviously "myScript.py" can be named whatever.

Next, create a directory for your models.

We'll name our model directory "myApp", but realize that this is a normal Django application within a project, as such, name it appropriately to the collection of models you are writing.

Within this directory create 2 files:

__init__.py
models.py


Your going to need a copy of manage.py from an either an existing Django project or you can just grab a copy from your Django install path:

djangoconfproject_templatemanage.py


Copy the manage.py to your /standAlone directory. Ok so you should now have the following structure:

standAlone
__init__.py
myScript.py
manage.py
settings.py
myApp
__init__.py
models.py


Add the following to your myScript.py file:

# settings.py
from django.conf import settings

settings.configure(
DATABASE_ENGINE = "postgresql_psycopg2",
DATABASE_NAME = "myDatabase",
DATABASE_USER = "myUsername",
DATABASE_PASSWORD = "myPassword",
DATABASE_HOST = "localhost",
DATABASE_PORT = "5432",
INSTALLED_APPS = ("myApp")
)

from django.db import models
from myApp.models import *


and add this to your settings.py file:

DATABASE_ENGINE = "postgresql_psycopg2"
DATABASE_NAME = "myDatabase"
DATABASE_USER = "myUsername"
DATABASE_PASSWORD = "myPassword"
DATABASE_HOST = "localhost"
DATABASE_PORT = "5432",
INSTALLED_APPS = ("myApp")


and finally your myApp/models.py:

# myApp/models.py
from django.db import models

class MyModel(models.Model):
field = models.CharField(max_length=255)


and that's it. Now to have Django manage your database, in command prompt navigate to our /standalone directory and run:

manage.py sql MyApp


I recommend SQLAlchemy. It should do all the ORM stuff, as well as basic SQL stuff.

I'm using django ORM without a settings file. Here's how:

In the stand-alone app launcher file:

from django.conf import settings
from django.core.management import execute_from_command_line

#Django settings
settings.configure(DEBUG=False,
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/path/to/dbfile',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
},
INSTALLED_APPS = ('modelsapp',)
)

if not os.path.exists('/path/to/dbfile'):
sync = ['manage.py', 'syncdb']
execute_from_command_line(sync)


Now you just need a ./modelsapp folder containing an __init__.py and a models.py. The config uses sqlite for simplicity sake, but it could use any of the db backends.

./modelsapp

__init__.py

models.py

Folder structure:

./launcher.py
./modelsapp
__init__.py
models.py


Note that you don't have to have a manage.py proper. The import execute_from_command_line just finds it.

import execute_from_command_line

Take a look at django-standalone which makes this setup pretty easy.

I also found this blog entry quite useful.

This example is as simple as it gets. I already have a django app called thab up and running. I want to use the django orm in free standing python scripts and use the same models as I'm using for web programming. Here is an example:

# nothing in my sys.path contains my django project files
import sys
sys.path.append('c:\apython\thab') # location of django app (module) called thab where my settings.py and models.py is
# my settings.py file is actualy in c:apythonthabthab
from thab import settings as s # need it because my database setting are there
dbs = s.DATABASES
from django.conf import settings
settings.configure(DATABASES=dbs) # configure can only be called once
from thab.models import *
boards = Board.objects.all()
print 'all boards:' + str(boards) # show all the boards in my board table


Probably I'm quite late with my answer, but it's better late than never.

Try this simple package:
https://github.com/serglopatin/django-models-standalone

How to use:

download

install

python setup.py install


create project

django-models-standalone startproject myproject


adjust files settings.py (DATABASES) and models.py, then migrate if tables not created

use djando models in your application (example.py)

import django
from django.conf import settings
from backend_mock.admin import settings as s
settings.configure(
DATABASES=s.DATABASES,
INSTALLED_APPS=('backend_mock.admin.mocker', )
)
django.setup()


take a look at this, it's working for django version gte 1.8.x

This is what worked for me in Django > 1.4

Assuming that your standalone script is your django project DIR.

Just copy this in a conf.py file (you can give it any name).

import os
import sys
import django

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)
#sys.path.append('c:\apython\thab')
# location of django app (module) called thab where my settings.py and models.py is
# my settings.py file is actualy in c:apythonthabthab
from elsaserver import settings as s # need it because my database setting are there
dbs = s.DATABASES
from django.conf import settings
settings.configure(
DATABASES=dbs,
INSTALLED_APPS=('core.apps.CoreConfig', )) #add all the apps you need here
django.setup()


Then import the conf.py in your python script.

This is the project structure:

mydjangoproject
|
> app1
> core
> app2
> standalone
| | __init__.py
| | conf.py
| | myscript.py
> manage.py


Can I use just the ORM part, to get classes that map to DB tables and know how to read/write from these tables?

Yes you can.

Here's a clean and short explanation on how to use Django's models and database abstraction: https://stackoverflow.com/a/49515366/2682613

Django version: 2.0.2

You can use it outside a django project.
But there are there things you should be aware of.

A router looks like:

class Router(object):
app_label = ''

def db_for_read(self, model, **hints):
if model._meta.app_label == self.app_label:
return self.app_label
return None

def db_for_write(self, model, **hints):
if model._meta.app_label == self.app_label:
return self.app_label
return None

def allow_relation(self, obj1, obj2, **hints):
if obj1._meta.app_label == self.app_label or obj2._meta.app_label == self.app_label:
return True
return None

def allow_migrate(self, db, app_label, model=None, **hints):
if app_label == self.app_label:
return db == self.app_label
return None


You can use meta class to dynamic create router.

def add_db(db_conf):
app_label = 'al_' + str(uuid4())

settings.DATABASES[app_label] = db_conf

router_class_name = 'Router' + app_label.capitalize()
setattr(
settings,
router_class_name,
type(router_class_name, (Router,), dict(app_label=app_label)),
)
settings.DATABASE_ROUTERS.append(
'.'.join([settings.__name__, router_class_name])
)
connections.close_all()
return app_label


the most important key is TIME_ZONE. DatetimeField and DateField is releated to it.
A simplest setting should be:

TIME_ZONE

DatetimeField

DateField

SECRET_KEY = 'secret'
DATABASES = {'default': {}}
DATABASE_ROUTERS =
TIME_ZONE = None


close_old_connections

Django framework default run close_old_connections in every request middleware to avoid "mysql has gone away".

PS: I have written package to use django orm not in a classic django project, https://github.com/jdxin0/django_db(https://github.com/jdxin0/django_db).
But you should always pay attention to the three problems above.My package use meta class to solve multi db, set TIME_ZONE=None and leave close_old_connections to user.

TIME_ZONE=None

close_old_connections

Using Django 2.0 ORM - One File Required

from myproject.config import parse_config
from django import setup as django_setup
from django.conf import settings as django_settings

"""
Requirements:
ODBC Driver: https://www.microsoft.com/en-ca/download/details.aspx?id=36434
Django Engine: https://pypi.org/project/django-pyodbc-azure/
"""

config = parse_config()
django_settings.configure(
DEBUG=True,
DATABASES={
'default': {
'ENGINE': 'sql_server.pyodbc',
'NAME': config.database_name,
'HOST': config.database_server, # exclude '\MSSQLSERVER'
'USER': config.database_username,
'PASSWORD': config.database_password,
'PORT': '',
'AUTOCOMMIT': False,
'OPTIONS': {
'driver': 'ODBC Driver 11 for SQL Server',
},
},
})
django_setup()


from django.db import models

class Foo(models.Model):
name = models.CharField(max_length=25)

class Meta:
app_label = 'myapp' # each model will require this


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.