A lot of Problems with Migrating (conceptual)

[Alexander Joseph]

I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.

I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)

More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and
data.

I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks

[Antonis Christofides]

Hello,

My opinion is that you should definitely switch to PostgreSQL, although I'm not
aware whether this would make migrating easier. I've written a relatively long
post
<https://djangodeployment.com/2016/12/23/which-database-should-i-use-on-production/>
about which database to choose in production.

Second, just to make things clear, the word "migration" has two meanings. The
original meaning of migration is to switch to another software system (e.g.
migrate from MySQL to PostgreSQL, or migrate a repository from subversion to
git). In Django, the term "migration" means something else: to update the
database to the new schema. In my opinion this terminology is wrong and
confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a
better one would have been "dbupdate" or something, but since it's migration
we'll keep it and you'll have to understand which meaning is meant in each case.

What your problem is cannot be understood with your general exposition, however
after you gain a little bit of experience with migrations you shouldn't have any
such problems. Migrations are tricky to understand but once you do they work
flawlessly. The next time you have a problem give us some information with the
exact error message so that we can explain what's wrong.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

Post by Alexander Joseph
I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.
I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my problems
stem from changing table fields, adding new tables/models, and migrating both
on my development server and my production server (PythonAnywhere)
More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and data.
I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Alexander Joseph]

Thanks Antonis, and great post on production databases. I'll definately be
switching to postgresql. I come from php/xamp and I liked the MySQL GUI
with phpMyAdmin so I've gravitated towards MySQL since.



On Thursday, August 17, 2017 at 2:05:44 PM UTC-6, Antonis Christofides

Post by Antonis Christofides
Hello,
My opinion is that you should definitely switch to PostgreSQL, although
I'm not aware whether this would make migrating easier. I've written a relatively
long post
<https://djangodeployment.com/2016/12/23/which-database-should-i-use-on-production/>
about which database to choose in production.
Second, just to make things clear, the word "migration" has two meanings.
The original meaning of migration is to switch to another software system
(e.g. migrate from MySQL to PostgreSQL, or migrate a repository from
to update the database to the new schema. In my opinion this terminology is
wrong and confusing (apparently it comes from Ruby on Rails, but I'm not
certain), and a better one would have been "dbupdate" or something, but
since it's migration we'll keep it and you'll have to understand which
meaning is meant in each case.
What your problem is cannot be understood with your general exposition,
however after you gain a little bit of experience with migrations you
shouldn't have any such problems. Migrations are tricky to understand but
once you do they work flawlessly. The next time you have a problem give us
some information with the exact error message so that we can explain what's
wrong.
Regards,
Antonis
Antonis Christofideshttp://djangodeployment.com
I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.
I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)
More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and data.
I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/16d60ab0-51e8-42a3-8f42-f4753f0adda3%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[James Bennett]

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <

Post by Antonis Christofides
Second, just to make things clear, the word "migration" has two meanings.
The original meaning of migration is to switch to another software system
(e.g. migrate from MySQL to PostgreSQL, or migrate a repository from
to update the database to the new schema. In my opinion this terminology is
wrong and confusing (apparently it comes from Ruby on Rails, but I'm not
certain), and a better one would have been "dbupdate" or something, but
since it's migration we'll keep it and you'll have to understand which
meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does
*not* only track the database schema. It also tracks the state of the
Python classes which define the models, including options which do not
affect the underlying DB schema, and making such changes to a model will
create the need for a migration.

This is absolutely necessary, though, because the migration framework
generates a frozen snapshot of the state at each migration, so that you can
access the ORM to do things like data updates via the RunPython migration
operator. If you were to change, say, the default ordering of a model
without also generating a migration to record that change (even though it
has no effect on the database schema), you could run into trouble if you
later removed a field referenced by the old ordering, since one of the
intermediate ORM snapshots would attempt to order queries by a
now-nonexistent column.

[Jim Illback]

Here’s a “problem” with migrations that I’ve had. It only occurs on a first time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table (Category):
category = forms.TypedChoiceField(required=False, choices=[(category.pk, category.name) for category in Category.objects.all()])

Running makemigrations or migrate if there’s no pre-existing database with a table named “Category” gives an error saying as much. However, I’m expecting migrate to create the tables required. Therefore, it’s a catch-22.

Of course, the “easy” answer is to comment out the above code, add in:
category = forms.CharField(max_length=12)
And run the initial migrations. Then, turn around and comment out that same code, uncomment the real code shown above, and run migrations again. It works, but it seems a bit awkward.

Has anyone else had this issue? Or am I doing something wrong?

Thanks much,
Jim Illback


On Aug 18, 2017, at 1:21 AM, James Bennett <***@gmail.com<mailto:***@gmail.com>> wrote:

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <***@djangodeployment.com<mailto:***@djangodeployment.com>> wrote:

Second, just to make things clear, the word "migration" has two meanings. The original meaning of migration is to switch to another software system (e.g. migrate from MySQL to PostgreSQL, or migrate a repository from subversion to git). In Django, the term "migration" means something else: to update the database to the new schema. In my opinion this terminology is wrong and confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a better one would have been "dbupdate" or something, but since it's migration we'll keep it and you'll have to understand which meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does *not* only track the database schema. It also tracks the state of the Python classes which define the models, including options which do not affect the underlying DB schema, and making such changes to a model will create the need for a migration.

This is absolutely necessary, though, because the migration framework generates a frozen snapshot of the state at each migration, so that you can access the ORM to do things like data updates via the RunPython migration operator. If you were to change, say, the default ordering of a model without also generating a migration to record that change (even though it has no effect on the database schema), you could run into trouble if you later removed a field referenced by the old ordering, since one of the intermediate ORM snapshots would attempt to order queries by a now-nonexistent column.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+***@googlegroups.com<mailto:django-users+***@googlegroups.com>.
To post to this group, send email to django-***@googlegroups.com<mailto:django-***@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com<https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[knbk]

There are various reasons why you should never execute queries at startup.
One you've seen: it makes it impossible to migrate a new database. Other
reasons include that the choices are cached and are only renewed when the
server restarts, and that when running tests, the query is executed before
the test database is set up and will run on the main database.

The solution in this case is to use a ModelChoiceField[1], i.e.:

category = forms.ModelChoiceField(queryset=Category.objects.all(),
required=False)

This allows to pass in a queryset which is evaluated whenever the field is
used to make sure the values are up-to-date. Since querysets are lazy, this
won't execute any queries on startup.

[1]
https://docs.djangoproject.com/en/1.11/ref/forms/fields/#modelchoicefield

Post by Jim Illback
Here’s a “problem” with migrations that I’ve had. It only occurs on a
first time installation (like a first-time upload to Heroku, for example).
category = forms.TypedChoiceField(required=False, choices=[(
category.pk, category.name) for category in Category.objects.all()])
Running makemigrations or migrate if there’s no pre-existing database with
a table named “Category” gives an error saying as much. However, I’m
expecting migrate to create the tables required. Therefore, it’s a
catch-22.
category = forms.CharField(max_length=12)
And run the initial migrations. Then, turn around and comment out that
same code, uncomment the real code shown above, and run migrations again.
It works, but it seems a bit awkward.
Has anyone else had this issue? Or am I doing something wrong?
Thanks much,
Jim Illback
On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <

Post by Antonis Christofides
Second, just to make things clear, the word "migration" has two meanings.
The original meaning of migration is to switch to another software system
(e.g. migrate from MySQL to PostgreSQL, or migrate a repository from
to update the database to the new schema. In my opinion this terminology is
wrong and confusing (apparently it comes from Ruby on Rails, but I'm not
certain), and a better one would have been "dbupdate" or something, but
since it's migration we'll keep it and you'll have to understand which
meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does
*not* only track the database schema. It also tracks the state of the
Python classes which define the models, including options which do not
affect the underlying DB schema, and making such changes to a model will
create the need for a migration.
This is absolutely necessary, though, because the migration framework
generates a frozen snapshot of the state at each migration, so that you can
access the ORM to do things like data updates via the RunPython migration
operator. If you were to change, say, the default ordering of a model
without also generating a migration to record that change (even though it
has no effect on the database schema), you could run into trouble if you
later removed a field referenced by the old ordering, since one of the
intermediate ORM snapshots would attempt to order queries by a
now-nonexistent column.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com
<https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Jim Illback]

Excellent - thanks much.

On Aug 18, 2017, at 4:02 PM, knbk <***@gmail.com<mailto:***@gmail.com>> wrote:

There are various reasons why you should never execute queries at startup. One you've seen: it makes it impossible to migrate a new database. Other reasons include that the choices are cached and are only renewed when the server restarts, and that when running tests, the query is executed before the test database is set up and will run on the main database.

The solution in this case is to use a ModelChoiceField[1], i.e.:

category = forms.ModelChoiceField(queryset=Category.objects.all(), required=False)

This allows to pass in a queryset which is evaluated whenever the field is used to make sure the values are up-to-date. Since querysets are lazy, this won't execute any queries on startup.

[1] https://docs.djangoproject.com/en/1.11/ref/forms/fields/#modelchoicefield

On Saturday, August 19, 2017 at 12:14:48 AM UTC+2, subaru_87 wrote:
Here’s a “problem” with migrations that I’ve had. It only occurs on a first time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table (Category):
category = forms.TypedChoiceField(required=False, choices=[(category.pk<http://category.pk/>, category.name<http://category.name/>) for category in Category.objects.all()])

Running makemigrations or migrate if there’s no pre-existing database with a table named “Category” gives an error saying as much. However, I’m expecting migrate to create the tables required. Therefore, it’s a catch-22.

Of course, the “easy” answer is to comment out the above code, add in:
category = forms.CharField(max_length=12)
And run the initial migrations. Then, turn around and comment out that same code, uncomment the real code shown above, and run migrations again. It works, but it seems a bit awkward.

Has anyone else had this issue? Or am I doing something wrong?

Thanks much,
Jim Illback


On Aug 18, 2017, at 1:21 AM, James Bennett <***@gmail.com<javascript:>> wrote:

On Thu, Aug 17, 2017 at 1:03 PM, Antonis Christofides <***@djangodeployment.com<javascript:>> wrote:

Second, just to make things clear, the word "migration" has two meanings. The original meaning of migration is to switch to another software system (e.g. migrate from MySQL to PostgreSQL, or migrate a repository from subversion to git). In Django, the term "migration" means something else: to update the database to the new schema. In my opinion this terminology is wrong and confusing (apparently it comes from Ruby on Rails, but I'm not certain), and a better one would have been "dbupdate" or something, but since it's migration we'll keep it and you'll have to understand which meaning is meant in each case.

An important thing worth knowing is that Django's migration framework does *not* only track the database schema. It also tracks the state of the Python classes which define the models, including options which do not affect the underlying DB schema, and making such changes to a model will create the need for a migration.

This is absolutely necessary, though, because the migration framework generates a frozen snapshot of the state at each migration, so that you can access the ORM to do things like data updates via the RunPython migration operator. If you were to change, say, the default ordering of a model without also generating a migration to record that change (even though it has no effect on the database schema), you could run into trouble if you later removed a field referenced by the old ordering, since one of the intermediate ORM snapshots would attempt to order queries by a now-nonexistent column.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-***@googlegroups.com<javascript:>.
To post to this group, send email to ***@googlegroups.com<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com<https://groups.google.com/d/msgid/django-users/CAL13Cg_0p1pMtMR2y2eGry1UhQaSRpswntcnMWtHNdPgV1Ph9w%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+***@googlegroups.com<mailto:django-users+***@googlegroups.com>.
To post to this group, send email to django-***@googlegroups.com<mailto:django-***@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d878688d-00a3-452d-8cb8-956dcf41a52e%40googlegroups.com<https://groups.google.com/d/msgid/django-users/d878688d-00a3-452d-8cb8-956dcf41a52e%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[James Schneider]

On Aug 18, 2017 3:14 PM, "Jim Illback" <***@hotmail.com> wrote:

Here’s a “problem” with migrations that I’ve had. It only occurs on a first
time installation (like a first-time upload to Heroku, for example).

In forms.py, I have a field dependent upon a column in another table
(Category):
category = forms.TypedChoiceField(required=False, choices=[(category.pk,
category.name) for category in Category.objects.all()])


While the ModelChoiceField is more applicable to this issue, an alternative
for other scenarios is to use a callback function as the value for
choices=, for example:

#early in forms.py
def get_categories():
return [(category.pk, category.name) for category in
Category.objects.all()]

# later in forms.py
category = forms.TypedChoiceField(required=False, choices=get_categories)

This has a similar effect to the ModelChoiceField, with the query only
running when the form is instantiated, but gives you the flexibility to use
whatever logic you'd like to come up with the choices for the field.

-James

[James Schneider]

# later in forms.py
category = forms.TypedChoiceField(required=False, choices=get_categories)

This has a similar effect to the ModelChoiceField, with the query only
running when the form is instantiated, but gives you the flexibility to use
whatever logic you'd like to come up with the choices for the field.


Forgot the doc link:


https://docs.djangoproject.com/en/1.11/ref/forms/fields/#django.forms.ChoiceField.choices

-James

[James Schneider]

On Aug 17, 2017 10:10 AM, "Alexander Joseph" <***@gmail.com>
wrote:

I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.

I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)

More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and
data.

I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks


Switching to a different DBMS will probably not alleviate migration issues,
especially if your project is still in its infancy and your migrations are
all still auto-generated. The migration system was designed to abstract
away the differences between the supported DBMS' so that you as the
developer didn't need to worry about it. Ideally you would be able to
deploy the same set of migrations on any of the supported databases.

Many Django devs have a bias towards Postgres, but much effort is put
toward ensuring that MySQL is supported since that is the only option for
many developers on their hosting platform. Some operations require extra
work in the migrations, and are challenging to implement regardless of the
DB platform. This is likely what you are running into.

I highly doubt MySQL itself is the cause, but it's possible. I'd recommend
posting up some of the errors that you are getting so that the folks here
can help. Might be more beneficial than setting up Postgres and having the
same issues.

-James

[Alexander Joseph]

Thanks for all the advice.

One more question - could project structure be causing issues with
migrations? I'm working on a large project and many apps in my project have
several layers of "sub-apps". My structure looks something like this...



my_project/
config/
docs/
accounting/
invoices/
__init__.py
admin.py
models.py
urls.py
views.py
purchase_orders/
__init__.py
admin.py
models.py
urls.py
views.py
__init__.py
admin.py
models.py
urls.py
views.py
engineering/
products/
product1/
sub_product1/
__init__.py
admin.py
models.py
urls.py
views.py
__init__.py
admin.py
models.py
urls.py
views.py
product2/
__init__.py
admin.py
models.py
urls.py
views.py
__init__.py
admin.py
models.py
urls.py
views.py
requirements/
utility/


A few things about my structure - I read in "2 Scoops of Django" that, in
general, if you have more than 5 models per model file then you should
think about splitting the model up into different apps, rather than having
long models files. And I structured it this way so that it would be a
little easier to manage - instead of having all apps under the project
folder, engineering apps would belong in the engineering folder, etc.

Thanks again

Post by Alexander Joseph
I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.
I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)
More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont fly
for long on production though of course .. once I actually have users and
data.
I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks

[James Schneider]

On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <

Post by Alexander Joseph
Thanks for all the advice.
One more question - could project structure be causing issues with
migrations? I'm working on a large project and many apps in my project have
several layers of "sub-apps". My structure looks something like this...

Possibly, especially if you're introducing any import loops. It's easy to
do. Trying to get the autodiscover mechanism for models to run through
nested apps can also be fun.

<snip>

A few things about my structure - I read in "2 Scoops of Django" that, in

Post by Alexander Joseph
general, if you have more than 5 models per model file then you should
think about splitting the model up into different apps, rather than having
long models files. And I structured it this way so that it would be a
little easier to manage - instead of having all apps under the project
folder, engineering apps would belong in the engineering folder, etc.
Thanks again

It depends. Your structure will likely change throughout the life of your
project. Start with what you have now and mold accordingly. Loose coupling
is key, though, to make it easier to move things around.

Like I mentioned in your other thread, you need to redefine what you are
considering an 'app'. Your current definition is way too specific/narrow,
which is why you are ending up with so many. Almost certainly each product
does not need its own app. That's not to say you shouldn't break them
apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
etc.) and break up the models into importable Python modules with a
sensible structure. Using modules will alleviate much of the pain of the
app and model discovery process using the pattern I described in the other
thread. It also provides a mechanism for loose coupling, and keeps your
models in separate files/folders using any structure you'd like. Reducing
the number of apps also reduces the number of files present in your
project, since you only need a single apps.py for a larger app, rather than
managing 20 apps.py files for 20 smaller apps.

2SoD is an excellent resource, and in no way to do I claim to have a
modicum of the knowledge and experience that the authors do. I don't even
program for a living. I have a copy for 1.8. You will notice in the book
that they do not have separate apps for an ice cream cone vs. ice cream
sandwiches vs. toppings, even though these may be analogous to your
product_1, product_2, etc. It's hard to tell though, because I'm assuming
you generalized your structure, so I don't know if you are talking about
different ice cream-based treat types, or cars vs. rocket ships. The latter
would likely use separate apps because there would be little overlapping
workflows and logic, along with multiple models associated with each.

I'd recommend looking in to some introductory material on database
normalization, not so much for the mechanics, but rather more conceptually.
The way you would break things apart for normalization often coincides
neatly with the way you should break apart apps (and builds your model list
for you), somewhere around the 2nd normal form. Again, your structure will
likely change as you go, so use the loose coupling via the __init__.py
module imports as I've shown, and that should help with the transition.

-James

[Alexander Joseph]

Thanks James, and thanks for your post on my other thread about structure.
I agree I need to get more into conceptual introductory material. I think I
will restructure my project and merge a lot of the apps/subapps I have now
as you suggested. From what Tom said it sounds like that could be a lot of
where my migration issues are coming from. I've seen a lot of dependency
related errors when trying to run migrations.

Thanks all for the ongoing advice. I'll try to keep this thread updated on
how issues are resolved as I restructure my project according to your
suggestions

Post by James Schneider
On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <

Post by Alexander Joseph
Thanks for all the advice.
One more question - could project structure be causing issues with
migrations? I'm working on a large project and many apps in my project have
several layers of "sub-apps". My structure looks something like this...

Possibly, especially if you're introducing any import loops. It's easy to
do. Trying to get the autodiscover mechanism for models to run through
nested apps can also be fun.
<snip>
A few things about my structure - I read in "2 Scoops of Django" that, in

Post by Alexander Joseph
general, if you have more than 5 models per model file then you should
think about splitting the model up into different apps, rather than having
long models files. And I structured it this way so that it would be a
little easier to manage - instead of having all apps under the project
folder, engineering apps would belong in the engineering folder, etc.
Thanks again

It depends. Your structure will likely change throughout the life of your
project. Start with what you have now and mold accordingly. Loose coupling
is key, though, to make it easier to move things around.
Like I mentioned in your other thread, you need to redefine what you are
considering an 'app'. Your current definition is way too specific/narrow,
which is why you are ending up with so many. Almost certainly each product
does not need its own app. That's not to say you shouldn't break them
apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
etc.) and break up the models into importable Python modules with a
sensible structure. Using modules will alleviate much of the pain of the
app and model discovery process using the pattern I described in the other
thread. It also provides a mechanism for loose coupling, and keeps your
models in separate files/folders using any structure you'd like. Reducing
the number of apps also reduces the number of files present in your
project, since you only need a single apps.py for a larger app, rather than
managing 20 apps.py files for 20 smaller apps.
2SoD is an excellent resource, and in no way to do I claim to have a
modicum of the knowledge and experience that the authors do. I don't even
program for a living. I have a copy for 1.8. You will notice in the book
that they do not have separate apps for an ice cream cone vs. ice cream
sandwiches vs. toppings, even though these may be analogous to your
product_1, product_2, etc. It's hard to tell though, because I'm assuming
you generalized your structure, so I don't know if you are talking about
different ice cream-based treat types, or cars vs. rocket ships. The latter
would likely use separate apps because there would be little overlapping
workflows and logic, along with multiple models associated with each.
I'd recommend looking in to some introductory material on database
normalization, not so much for the mechanics, but rather more conceptually.
The way you would break things apart for normalization often coincides
neatly with the way you should break apart apps (and builds your model list
for you), somewhere around the 2nd normal form. Again, your structure will
likely change as you go, so use the loose coupling via the __init__.py
module imports as I've shown, and that should help with the transition.
-James
--
You received this message because you are subscribed to a topic in the
Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/django-users/kzNqB9lEIGA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pymp
BPK2btm38FshX%3DSX1Q%40mail.gmail.com
<https://groups.google.com/d/msgid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pympBPK2btm38FshX%3DSX1Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Best Regards,

Alexander Joseph

[Alexander Joseph]

One more question - is there a way to put apps in folders/sub-folders
instead of creating sub-apps/modules? I just want to keep things easier to
navigate on the development side. I will eventually have about 20 sub-apps
in the 'Engineering' app alone.

Even if I could just group all the engineering sub-apps i have now under an
engineering folder without any further hierarchy that would help as there
will also be HR, financial apps, administration apps, etc.

Thanks again



On Wed, Aug 23, 2017 at 5:28 PM, Alexander Joseph <

Post by Alexander Joseph
Thanks James, and thanks for your post on my other thread about structure.
I agree I need to get more into conceptual introductory material. I think I
will restructure my project and merge a lot of the apps/subapps I have now
as you suggested. From what Tom said it sounds like that could be a lot of
where my migration issues are coming from. I've seen a lot of dependency
related errors when trying to run migrations.
Thanks all for the ongoing advice. I'll try to keep this thread updated on
how issues are resolved as I restructure my project according to your
suggestions

Post by James Schneider
On Tue, Aug 22, 2017 at 12:37 PM, Alexander Joseph <

Post by Alexander Joseph
Thanks for all the advice.
One more question - could project structure be causing issues with
migrations? I'm working on a large project and many apps in my project have
several layers of "sub-apps". My structure looks something like this...

Possibly, especially if you're introducing any import loops. It's easy to
do. Trying to get the autodiscover mechanism for models to run through
nested apps can also be fun.
<snip>
A few things about my structure - I read in "2 Scoops of Django" that, in

Post by Alexander Joseph
general, if you have more than 5 models per model file then you should
think about splitting the model up into different apps, rather than having
long models files. And I structured it this way so that it would be a
little easier to manage - instead of having all apps under the project
folder, engineering apps would belong in the engineering folder, etc.
Thanks again

It depends. Your structure will likely change throughout the life of your
project. Start with what you have now and mold accordingly. Loose coupling
is key, though, to make it easier to move things around.
Like I mentioned in your other thread, you need to redefine what you are
considering an 'app'. Your current definition is way too specific/narrow,
which is why you are ending up with so many. Almost certainly each product
does not need its own app. That's not to say you shouldn't break them
apart. I'm saying keep your 'apps' generalized (ie engineering, accounting,
etc.) and break up the models into importable Python modules with a
sensible structure. Using modules will alleviate much of the pain of the
app and model discovery process using the pattern I described in the other
thread. It also provides a mechanism for loose coupling, and keeps your
models in separate files/folders using any structure you'd like. Reducing
the number of apps also reduces the number of files present in your
project, since you only need a single apps.py for a larger app, rather than
managing 20 apps.py files for 20 smaller apps.
2SoD is an excellent resource, and in no way to do I claim to have a
modicum of the knowledge and experience that the authors do. I don't even
program for a living. I have a copy for 1.8. You will notice in the book
that they do not have separate apps for an ice cream cone vs. ice cream
sandwiches vs. toppings, even though these may be analogous to your
product_1, product_2, etc. It's hard to tell though, because I'm assuming
you generalized your structure, so I don't know if you are talking about
different ice cream-based treat types, or cars vs. rocket ships. The latter
would likely use separate apps because there would be little overlapping
workflows and logic, along with multiple models associated with each.
I'd recommend looking in to some introductory material on database
normalization, not so much for the mechanics, but rather more conceptually.
The way you would break things apart for normalization often coincides
neatly with the way you should break apart apps (and builds your model list
for you), somewhere around the 2nd normal form. Again, your structure will
likely change as you go, so use the loose coupling via the __init__.py
module imports as I've shown, and that should help with the transition.
-James
--
You received this message because you are subscribed to a topic in the
Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/django-users/kzNqB9lEIGA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pympBPK2bt
m38FshX%3DSX1Q%40mail.gmail.com
<https://groups.google.com/d/msgid/django-users/CA%2Be%2BciV7tZS8WJeFyn1p5MQrctM6pympBPK2btm38FshX%3DSX1Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Best Regards,
Alexander Joseph

--
Best Regards,

Alexander Joseph

[James Schneider]

On Wed, Aug 23, 2017 at 4:40 PM, Alexander Joseph <

Post by Alexander Joseph
One more question - is there a way to put apps in folders/sub-folders
instead of creating sub-apps/modules? I just want to keep things easier to
navigate on the development side. I will eventually have about 20 sub-apps
in the 'Engineering' app alone.
Even if I could just group all the engineering sub-apps i have now under
an engineering folder without any further hierarchy that would help as
there will also be HR, financial apps, administration apps, etc.
Thanks again

Technically you can go as deep as you'd like. You can use the module
strategy to emulate a Django 'app' without all the registration fuss. A
couple abbreviated examples:


# 1. Group resources and references by object type
project/
engineering/
__init__.py
models/
__init__.py
widgets.py
gadgets.py
product1.py
product2.py
views/
__init__.py
product1.py
product2.py
# etc.

All of the models for engineering would be grouped in a single 'models'
module. Imports would look like 'from engineering.models.widgets import
WonderWidget'

The engineering/models/__init__.py file would contain lines like 'from
.widgets import *' for all of your files that contain models.


# 2. Group by business segment or workflow
project/
engineering/
__init__.py
models.py
product1/
__init__.py
models.py
views.py
urls.py
product2/
__init__.py
models.py
views.py
urls.py
# etc.

In this case, you're replicating the structure of an 'app' without creating
one by Django's definition. The engineering/models.py file would then
contain imports like 'from .product1.models import *' in order to get the
model auto-discovery to work correctly.

I'm under the impression that most developers use layout #1, but a large
project might work better with #2 if there aren't a lot of
cross-dependencies.

Note that using Python modules rather than real Django apps will also keep
your settings.INSTALLED_APPS list minimized. With as many 'apps' as you
were originally talking about, that list could be dozens or hundreds of
items long.

IMHO I would start with #1 and see how it works for you. It could be the
generalized term 'product' that you're using, but to me, a product would
not trigger a new 'app' for me, just extra model/view/url/etc. files,
especially given how often products change or are added/removed. In that
case, #2 might work better since all of the related code is within a single
sub-folder (but all the references to that code aren't, so there is still
work to be done). YMMV

-James

[Alexander Joseph]

Awesome, thanks James, thats exactly what I'm looking for. I'll try layout
1 first as you suggest

Post by James Schneider

Post by Alexander Joseph
One more question - is there a way to put apps in folders/sub-folders
instead of creating sub-apps/modules? I just want to keep things easier to
navigate on the development side. I will eventually have about 20 sub-apps
in the 'Engineering' app alone.
Even if I could just group all the engineering sub-apps i have now under
an engineering folder without any further hierarchy that would help as
there will also be HR, financial apps, administration apps, etc.
Thanks again

Technically you can go as deep as you'd like. You can use the module
strategy to emulate a Django 'app' without all the registration fuss. A
# 1. Group resources and references by object type
project/
engineering/
__init__.py
models/
__init__.py
widgets.py
gadgets.py
product1.py
product2.py
views/
__init__.py
product1.py
product2.py
# etc.
All of the models for engineering would be grouped in a single 'models'
module. Imports would look like 'from engineering.models.widgets import
WonderWidget'
The engineering/models/__init__.py file would contain lines like 'from
.widgets import *' for all of your files that contain models.
# 2. Group by business segment or workflow
project/
engineering/
__init__.py
models.py
product1/
__init__.py
models.py
views.py
urls.py
product2/
__init__.py
models.py
views.py
urls.py
# etc.
In this case, you're replicating the structure of an 'app' without
creating one by Django's definition. The engineering/models.py file would
then contain imports like 'from .product1.models import *' in order to get
the model auto-discovery to work correctly.
I'm under the impression that most developers use layout #1, but a large
project might work better with #2 if there aren't a lot of
cross-dependencies.
Note that using Python modules rather than real Django apps will also keep
your settings.INSTALLED_APPS list minimized. With as many 'apps' as you
were originally talking about, that list could be dozens or hundreds of
items long.
IMHO I would start with #1 and see how it works for you. It could be the
generalized term 'product' that you're using, but to me, a product would
not trigger a new 'app' for me, just extra model/view/url/etc. files,
especially given how often products change or are added/removed. In that
case, #2 might work better since all of the related code is within a single
sub-folder (but all the references to that code aren't, so there is still
work to be done). YMMV
-James

[Alexander Joseph]

OK, so I started over using postrgresql and so far I have one app named
'accounts' - to hold user models, registration, etc. I tried to migrate and
got the following error which looks a lot like some of the errors I was
getting before..


(business_management) C:\Python36\Projects\business_management>python
manage.py migrate --settings=config.settings.local
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\__init__.py",
line 363, in execute_from_command_line
utility.execute()
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\__init__.py",
line 355, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\base.py",
line 283, in run_from_argv
self.execute(*args, **cmd_options)
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\base.py",
line 330, in execute
output = self.handle(*args, **options)
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\core\management\commands\migrate.py",
line 86, in handle
executor.loader.check_consistent_history(connection)
File
"C:\Users\Alexander\Envs\business_management\lib\site-packages\django\db\migrations\loader.py",
line 298, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
admin.0001_initial is applied before its dependency accounts.0001_initial
on database 'default'.



let me know if you need anymore info to help me out with this. Thanks in
advance

Post by Alexander Joseph
Awesome, thanks James, thats exactly what I'm looking for. I'll try layout
1 first as you suggest

Post by James Schneider

Post by Alexander Joseph
One more question - is there a way to put apps in folders/sub-folders
instead of creating sub-apps/modules? I just want to keep things easier to
navigate on the development side. I will eventually have about 20 sub-apps
in the 'Engineering' app alone.
Even if I could just group all the engineering sub-apps i have now under
an engineering folder without any further hierarchy that would help as
there will also be HR, financial apps, administration apps, etc.
Thanks again

Technically you can go as deep as you'd like. You can use the module
strategy to emulate a Django 'app' without all the registration fuss. A
# 1. Group resources and references by object type
project/
engineering/
__init__.py
models/
__init__.py
widgets.py
gadgets.py
product1.py
product2.py
views/
__init__.py
product1.py
product2.py
# etc.
All of the models for engineering would be grouped in a single 'models'
module. Imports would look like 'from engineering.models.widgets import
WonderWidget'
The engineering/models/__init__.py file would contain lines like 'from
.widgets import *' for all of your files that contain models.
# 2. Group by business segment or workflow
project/
engineering/
__init__.py
models.py
product1/
__init__.py
models.py
views.py
urls.py
product2/
__init__.py
models.py
views.py
urls.py
# etc.
In this case, you're replicating the structure of an 'app' without
creating one by Django's definition. The engineering/models.py file would
then contain imports like 'from .product1.models import *' in order to get
the model auto-discovery to work correctly.
I'm under the impression that most developers use layout #1, but a large
project might work better with #2 if there aren't a lot of
cross-dependencies.
Note that using Python modules rather than real Django apps will also
keep your settings.INSTALLED_APPS list minimized. With as many 'apps' as
you were originally talking about, that list could be dozens or hundreds of
items long.
IMHO I would start with #1 and see how it works for you. It could be the
generalized term 'product' that you're using, but to me, a product would
not trigger a new 'app' for me, just extra model/view/url/etc. files,
especially given how often products change or are added/removed. In that
case, #2 might work better since all of the related code is within a single
sub-folder (but all the references to that code aren't, so there is still
work to be done). YMMV
-James

[James Schneider]

<snip>

File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\core\management\commands\migrate.py", line 86, in handle
executor.loader.check_consistent_history(connection)
File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\db\migrations\loader.py", line 298, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
admin.0001_initial is applied before its dependency accounts.0001_initial
on database 'default'.



let me know if you need anymore info to help me out with this. Thanks in
advance



How are you generating your migration files?

Are you specifying the app_label after the makemigrations command?

Are you modifying the migration files after they've been generated?

Do you have an app with a label of admin? Or is that a reference to the
built-in admin?

-James

[Alexander Joseph]

I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local" and "python manage.py
migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

Post by James Schneider
<snip>
File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\core\management\commands\migrate.py", line 86, in handle
executor.loader.check_consistent_history(connection)
File "C:\Users\Alexander\Envs\business_management\lib\site-packag
es\django\db\migrations\loader.py", line 298, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
admin.0001_initial is applied before its dependency accounts.0001_initial
on database 'default'.
let me know if you need anymore info to help me out with this. Thanks in
advance
How are you generating your migration files?
Are you specifying the app_label after the makemigrations command?
Are you modifying the migration files after they've been generated?
Do you have an app with a label of admin? Or is that a reference to the
built-in admin?
-James
--
You received this message because you are subscribed to a topic in the
Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/django-users/kzNqB9lEIGA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-users/CA%2Be%2BciUKFGrv62awj%2BuuSSuVaS9uYqVs2%2Brv%
3DJPsaxvmJGNTfg%40mail.gmail.com
<https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUKFGrv62awj%2BuuSSuVaS9uYqVs2%2Brv%3DJPsaxvmJGNTfg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Best Regards,

Alexander Joseph

[James Schneider]

On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph <

Post by Alexander Joseph
I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local" and "python manage.py
migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

I know you were doing a lot of refactoring. Did you rename the accounts app
after performing an initial migration (applying an old migration with the
old app name structure)? It sounds like your DB and your migration files
might be out of sync. If you don't have any production data, I would
recommend deleting and recreating the entire database, and deleting and
recreating your migrations so that all the migrations are run in the
correct order.

If you do have data you want to keep, then very likely you now have a ton
of work to do writing custom migrations to sync everything back up
manually.

-James

[Alexander Joseph]

Thanks James,
I'm actually starting over from scratch instead of actually refactoring so
the accounts app is the only real app I have so far. I actually did delete
the database and just apply migrations to the newly created database and it
worked ok. I'd like to get better at fixing these migration problems though
since I'll probably run into a lot of them and wont be able to delete the
database once I have data.

Do you think it would have been better to run migrations for the specific
accounts app? I thought it wouldnt matter much since accounts is my only
app I have so far.

My new structure so far looks like this

business_management/
business_management/
accounts/
migrations/
__init__.py
admin.py
forms.py
models.py
tests.py
views.py
static/
templates/
config/
settings/
__init__.py
base.py
local.py
production.py
__init__.py
docs/
(developer documentation)
requirements/
(requirements.txt files)
utility/
(shell scripts)
manage.py
.gitignore

Post by James Schneider

Post by Alexander Joseph
I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local" and "python manage.py
migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

I know you were doing a lot of refactoring. Did you rename the accounts
app after performing an initial migration (applying an old migration with
the old app name structure)? It sounds like your DB and your migration
files might be out of sync. If you don't have any production data, I would
recommend deleting and recreating the entire database, and deleting and
recreating your migrations so that all the migrations are run in the
correct order.
If you do have data you want to keep, then very likely you now have a ton
of work to do writing custom migrations to sync everything back up
manually.
-James

[Melvyn Sopacua]

There are two main principles to remember about Django migrations in early
development stage:

1) Make migrations often and coherent.
Typically, if you change something about a model, make a migration.
When you want to cover multiple models in one migration, there should be
a good reason for it. If you can't think of it, don't.

2) When the design is flawed, don't add to the problem by creating another
migration. Instead roll back, remove the migration file(s), alter the
design and make a new migration.

The second principle is often overlooked and by applying the first, the
"damage" is minimized.

Rolling back is done by adding a migration number to the `migrate`
management
command and it will roll back to the point including that migration.

Once you freeze the model design, feel free to squash some or all migrations
into one file if things got a little big.


On Wed, Aug 30, 2017 at 12:16 AM, Alexander Joseph <

Post by Alexander Joseph
Thanks James,
I'm actually starting over from scratch instead of actually refactoring so
the accounts app is the only real app I have so far. I actually did delete
the database and just apply migrations to the newly created database and it
worked ok. I'd like to get better at fixing these migration problems though
since I'll probably run into a lot of them and wont be able to delete the
database once I have data.
Do you think it would have been better to run migrations for the specific
accounts app? I thought it wouldnt matter much since accounts is my only
app I have so far.
My new structure so far looks like this
business_management/
business_management/
accounts/
migrations/
__init__.py
admin.py
forms.py
models.py
tests.py
views.py
static/
templates/
config/
settings/
__init__.py
base.py
local.py
production.py
__init__.py
docs/
(developer documentation)
requirements/
(requirements.txt files)
utility/
(shell scripts)
manage.py
.gitignore

Post by James Schneider

Post by Alexander Joseph
I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local" and "python
manage.py migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

I know you were doing a lot of refactoring. Did you rename the accounts
app after performing an initial migration (applying an old migration with
the old app name structure)? It sounds like your DB and your migration
files might be out of sync. If you don't have any production data, I would
recommend deleting and recreating the entire database, and deleting and
recreating your migrations so that all the migrations are run in the
correct order.
If you do have data you want to keep, then very likely you now have a ton
of work to do writing custom migrations to sync everything back up
manually.
-James

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Melvyn Sopacua

[Alexander Joseph]

Thanks Melvyn, good to know. I'll keep that in mind

Post by Melvyn Sopacua
There are two main principles to remember about Django migrations in early
1) Make migrations often and coherent.
Typically, if you change something about a model, make a migration.
When you want to cover multiple models in one migration, there should be
a good reason for it. If you can't think of it, don't.
2) When the design is flawed, don't add to the problem by creating another
migration. Instead roll back, remove the migration file(s), alter the
design and make a new migration.
The second principle is often overlooked and by applying the first, the
"damage" is minimized.
Rolling back is done by adding a migration number to the `migrate`
management
command and it will roll back to the point including that migration.
Once you freeze the model design, feel free to squash some or all migrations
into one file if things got a little big.

Post by Alexander Joseph
Thanks James,
I'm actually starting over from scratch instead of actually refactoring
so the accounts app is the only real app I have so far. I actually did
delete the database and just apply migrations to the newly created database
and it worked ok. I'd like to get better at fixing these migration problems
though since I'll probably run into a lot of them and wont be able to
delete the database once I have data.
Do you think it would have been better to run migrations for the specific
accounts app? I thought it wouldnt matter much since accounts is my only
app I have so far.
My new structure so far looks like this
business_management/
business_management/
accounts/
migrations/
__init__.py
admin.py
forms.py
models.py
tests.py
views.py
static/
templates/
config/
settings/
__init__.py
base.py
local.py
production.py
__init__.py
docs/
(developer documentation)
requirements/
(requirements.txt files)
utility/
(shell scripts)
manage.py
.gitignore

Post by James Schneider
On Tue, Aug 29, 2017 at 7:13 AM, Alexander Joseph <

Post by Alexander Joseph
I'm not specifying the app level, I'm just running "python manage.py
makemigrations --settings=config.settings.local" and "python
manage.py migrate --settings=config.settings.local"
I'm not modifying the migrations files and I dont have an app with the
label of admin, thats just the built-in admin. Thanks

I know you were doing a lot of refactoring. Did you rename the accounts
app after performing an initial migration (applying an old migration with
the old app name structure)? It sounds like your DB and your migration
files might be out of sync. If you don't have any production data, I would
recommend deleting and recreating the entire database, and deleting and
recreating your migrations so that all the migrations are run in the
correct order.
If you do have data you want to keep, then very likely you now have a
ton of work to do writing custom migrations to sync everything back up
manually.
-James

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/0686084f-1d5a-4f61-823e-7d84c58f317c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Melvyn Sopacua

[Antonis Christofides]

Hi,

I read in "2 Scoops of Django" that, in general, if you have more than 5
models per model file then you should think about splitting the model up into
different apps, rather than having long models files.

I don't claim to be more experienced than the authors—on the contrary—and I have
learnt loads from this book. However I disagree with this assertion. Five models
is very few. In addition, I've found early splitting to usually be YAGNI
<https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it>. It causes more
problems than it solves. Unless there is an obvious logical distinction of
functionality right from the start, put all your models in the same app. When it
grows and you see an obvious way to improve its structure, go on and refactor.
Don't "refactor" from the start before you even understand your app's structure.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

Thanks for all the advice.
One more question - could project structure be causing issues with migrations?
I'm working on a large project and many apps in my project have several layers
of "sub-apps". My structure looks something like this...
|
my_project/
    config/ 
    docs/
    accounting/
        invoices/
            __init__.py
            admin.py
            models.py
            urls.py
            views.py
        purchase_orders/
            __init__.py
            admin.py
            models.py
            urls.py
            views.py 
        __init__.py
        admin.py
        models.py
        urls.py
        views.py
    engineering/
        products/
            product1/
                sub_product1/
                    __init__.py
                    admin.py
                    models.py
                    urls.py
                    views.py
                __init__.py
                admin.py
                models.py
                urls.py
                views.py
            product2/
                __init__.py
                admin.py
                models.py
                urls.py
                views.py
            __init__.py
            admin.py
            models.py
            urls.py
            views.py
    requirements/
    utility/
|
A few things about my structure - I read in "2 Scoops of Django" that, in
general, if you have more than 5 models per model file then you should think
about splitting the model up into different apps, rather than having long
models files. And I structured it this way so that it would be a little easier
to manage - instead of having all apps under the project folder, engineering
apps would belong in the engineering folder, etc.
Thanks again
I'm pretty new to django and I've been having problems with
makemigrations/migrate and I wonder if Im doing things right.
I'm using MySQL backend and after reading in the documentation a little it
sounds like Postgresql might make migrating more painless. Usually my
problems stem from changing table fields, adding new tables/models, and
migrating both on my development server and my production server
(PythonAnywhere)
More than once I've had to drop my database, delete all my migrations, and
start over with initial migrations on the development server. This wont
fly for long on production though of course .. once I actually have users
and data.
I'm wondering if I should switch to postgresql or if theres any further
resources that you might know of that would help me out? Thanks
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/d4cc32e5-c9c1-48d7-bb19-a6646800ce3d%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/d4cc32e5-c9c1-48d7-bb19-a6646800ce3d%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

['Tom Evans' via Django users]

On Tue, Aug 22, 2017 at 8:37 PM, Alexander Joseph

Post by Alexander Joseph
Thanks for all the advice.
One more question - could project structure be causing issues with
migrations? I'm working on a large project and many apps in my project have
several layers of "sub-apps". My structure looks something like this...

This approach seems appealing, but personally I think it is a bad idea
because the ease of introducing dependencies between apps is extremely
high. Even when the apps are all tightly related to a particular
project (to the extent their package name is
"fooproject-some-function"), I think its better to simply develop each
one as a standalone app, separate from the other apps and not
requiring the other apps in order to test/develop.

With this approach, you will not get unnecessarily thin apps simply
because you cannot easily couple two apps together, and everything
within the app should have high cohesion. Remember (or discover :)
that in software engineering, high quality code is loosely coupled (it
doesn't depend on many external things) and highly cohesive (the
things within the module are related to each other).

I would also avoid the use of subapps. First off, Django only uses the
final part of the dotted import path as the app name -
'engineering.products.product1' has the same name as
'accounting.invoices.product1' (I know that doesn't exist, but...). I
have a (unsubstantiated) theory that your migration woes are down to
subapps.

Long model files are not good, but I'd also disagree with 2SoD that
getting to 5 models means a new app because of long files - what
ridiculousness! If your model file is getting long, replace it with a
model directory, each model in a separate module, and all of them
imported in the __init__. You should split your app when it is no
longer cohesive (or more accurately, when the level of cohesion is
upsetting and/or slows development). 2SoD is good, but its not a holy
text!

Cheers

Tom

PS

Make sure you have a sensible workflow for migrations - I'm assuming
you are using some sort of version control. While you are developing
your changes, you might generate a lot of temporary migrations. You do
not want to apply these migrations to live, so you should squash them
- either manually, using squashmigrations or by migrating the DB back
to before they existed, remove them and recreate them. If you do the
last one, remember to preserve any custom logic you have placed in the
migration.

At the end of the day, the migrations you commit are what will run on
your production database, so you should ensure they do exactly what
you are expecting them to do and are running in the correct order.
Particularly if you have a lot of small apps, getting the dependencies
right is essential.