Import Error Hotfix (#148)

* Changed uvicorn port to 80

* Changed port in docker-compose to match dockerfile

* Readded environment variables in docker-compose

* production image rework

* Use opengraph metadata to make basic recipe cards when full recipe metadata is not available

* fixed instrucitons on parse

* add last_recipe

* automated testing

* roadmap update

* Sqlite (#75)

* file structure

* auto-test

* take 2

* refactor ap scheduler and startup process

* fixed scraper error

* database abstraction

* database abstraction

* port recipes over to new schema

* meal migration

* start settings migration

* finale mongo port

* backup improvements

* migration imports to new DB structure

* unused import cleanup

* docs strings

* settings and theme import logic

* cleanup

* fixed tinydb error

* requirements

* fuzzy search

* remove scratch file

* sqlalchemy models

* improved search ui

* recipe models almost done

* sql modal population

* del scratch

* rewrite database model mixins

* mostly grabage

* recipe updates

* working sqllite

* remove old files and reorganize

* final cleanup

Co-authored-by: Hayden <hay-kot@pm.me>

* Backup card (#78)

* backup / import dialog

* upgrade to new tag method

* New import card

* rename settings.py to app_config.py

* migrate to poetry for development

* fix failing test

Co-authored-by: Hayden <hay-kot@pm.me>

* added mkdocs to docker-compose

* Translations (#72)

* Translations + danish

* changed back proxy target to use ENV

* Resolved more merge conflicts

* Removed test in translation

* Documentation of translations

* Updated translations

* removed old packages

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>

* fail to start bug fixes

* feature: prep/cook/total time slots (#80)

Co-authored-by: Hayden <hay-kot@pm.me>

* missing bind attributes

* Bug fixes (#81)

* fix: url remains after succesful import

* docs: changelog + update todos

* arm image

* arm compose

* compose updates

* update poetry

* arm support

Co-authored-by: Hayden <hay-kot@pm.me>

* dockerfile hotfix

* dockerfile hotfix

* Version Release Final Touches (#84)

* Remove slim

* bug: opacity issues

* bug: startup failure with no database

* ci/cd on dev branch

* formatting

* v0.1.0 documentation

Co-authored-by: Hayden <hay-kot@pm.me>

* db init hotfix

* bug: fix crash in mongo

* fix mongo bug

* fixed version notifier

* finale changelog

* Dropping Mongo From Dev Branch (#89)

* Fix link to Docker Hub

Found an extra s. DESTROYED it.

* initial pass

* second pass cleanup

* backup card framework

* backup card functionality

* translation

* upload button vile creation

* Release v0.1.0 Candidate (#85)

* Changed uvicorn port to 80

* Changed port in docker-compose to match dockerfile

* Readded environment variables in docker-compose

* production image rework

* Use opengraph metadata to make basic recipe cards when full recipe metadata is not available

* fixed instrucitons on parse

* add last_recipe

* automated testing

* roadmap update

* Sqlite (#75)

* file structure

* auto-test

* take 2

* refactor ap scheduler and startup process

* fixed scraper error

* database abstraction

* database abstraction

* port recipes over to new schema

* meal migration

* start settings migration

* finale mongo port

* backup improvements

* migration imports to new DB structure

* unused import cleanup

* docs strings

* settings and theme import logic

* cleanup

* fixed tinydb error

* requirements

* fuzzy search

* remove scratch file

* sqlalchemy models

* improved search ui

* recipe models almost done

* sql modal population

* del scratch

* rewrite database model mixins

* mostly grabage

* recipe updates

* working sqllite

* remove old files and reorganize

* final cleanup

Co-authored-by: Hayden <hay-kot@pm.me>

* Backup card (#78)

* backup / import dialog

* upgrade to new tag method

* New import card

* rename settings.py to app_config.py

* migrate to poetry for development

* fix failing test

Co-authored-by: Hayden <hay-kot@pm.me>

* added mkdocs to docker-compose

* Translations (#72)

* Translations + danish

* changed back proxy target to use ENV

* Resolved more merge conflicts

* Removed test in translation

* Documentation of translations

* Updated translations

* removed old packages

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>

* fail to start bug fixes

* feature: prep/cook/total time slots (#80)

Co-authored-by: Hayden <hay-kot@pm.me>

* missing bind attributes

* Bug fixes (#81)

* fix: url remains after succesful import

* docs: changelog + update todos

* arm image

* arm compose

* compose updates

* update poetry

* arm support

Co-authored-by: Hayden <hay-kot@pm.me>

* dockerfile hotfix

* dockerfile hotfix

* Version Release Final Touches (#84)

* Remove slim

* bug: opacity issues

* bug: startup failure with no database

* ci/cd on dev branch

* formatting

* v0.1.0 documentation

Co-authored-by: Hayden <hay-kot@pm.me>

* db init hotfix

* bug: fix crash in mongo

* fix mongo bug

* fixed version notifier

* finale changelog

Co-authored-by: kentora <=>
Co-authored-by: Hayden <hay-kot@pm.me>
Co-authored-by: Richard Mitic <richard.h.mitic@gmail.com>
Co-authored-by: kentora <kentora@kentora.dk>

* build container

* webscraper hotfix

* dev bug: change data location to prevent reloads

* api docs

* api docs bug

* workflow update

Co-authored-by: David Young <davidy@funkypenguin.co.nz>
Co-authored-by: Hayden <hay-kot@pm.me>
Co-authored-by: Richard Mitic <richard.h.mitic@gmail.com>
Co-authored-by: kentora <kentora@kentora.dk>

* Add French Translation (#93)

* New tests (#94)

* dev-bug: fixed vscode freezes

* test: refactor database init to support tests

Co-authored-by: Hayden <hay-kot@pm.me>

* Mealplan CRUD Tests (#95)

* dev-bug: fixed vscode freezes

* test: refactor database init to support tests

* mealplan CRUD testing

Co-authored-by: Hayden <hay-kot@pm.me>

* Fix typos (#96)

* Settings, Themes and Migration Route Tests (#100)

* dev-bug: fixed vscode freezes

* test: refactor database init to support tests

* mealplan CRUD testing

* restructure test folder

* git attributes

* tests: migration, settings, theme routes testing

Co-authored-by: Hayden <hay-kot@pm.me>

* Refactor + New Docker File (#105)

* dev-bug: fixed vscode freezes

* test: refactor database init to support tests

* mealplan CRUD testing

* restructure test folder

* git attributes

* tests: migration, settings, theme routes testing

* docker-file shrink

* rebuild

* refactor: moving directories around

* adding funding

Co-authored-by: Hayden <hay-kot@pm.me>

* Meal planner improvements (#107)

* dev-bug: fixed vscode freezes

* test: refactor database init to support tests

* mealplan CRUD testing

* restructure test folder

* git attributes

* tests: migration, settings, theme routes testing

* docker-file shrink

* rebuild

* refactor: moving directories around

* adding funding

* mealplan redesign

Co-authored-by: Hayden <hay-kot@pm.me>

* Upload component (#108)

* unified upload button + download backups

* javascript toolings

* fix vuetur config

* fixed type check error

* refactor: clean up bag javascript

Co-authored-by: Hayden <hay-kot@pm.me>

* Upload component (#113)

* unified upload button + download backups

* javascript toolings

* fix vuetur config

* fixed type check error

* refactor: clean up bag javascript

* UI updates + name validation

* docs: changelog + sp

* fixed route links

* changelog

Co-authored-by: Hayden <hay-kot@pm.me>

* fixed menu links

* fixed poetry install on docker.dev build

* Migration redesign (#119)

* migration redesign init

* new color picker

* changelog

* added UI language selection

* fix layout issue on recipe editor

* remove git as dependency

* added UI editor for original URL

* CI/CD Tests

* test: fixed migration routes

Co-authored-by: Hayden <hay-kot@pm.me>

* Fix link to dev-notes.md (#110)

* translation: add swedish (#128)

* language: da is Danish

* translations: add swedish

* scraper: unescape html in instructions (#129)

Some urls erroneously deliver escaped html their instructions,
sometimes they are even escaped on multiple levels like here:

https://www.ica.se/recept/kladdig-kladdkaka-722982/

```
>>> normalize_instruction("S&amp;auml;tt ugnen p&amp;aring; 200&amp;deg;C.")
'Sätt ugnen på 200°C.'
```

* v0.2.0 Updates (#130)

* migration redesign init

* new color picker

* changelog

* added UI language selection

* fix layout issue on recipe editor

* remove git as dependency

* added UI editor for original URL

* CI/CD Tests

* test: fixed migration routes

* test todos

* bug/added docker volume

* chowdow test data

* partial image recipe image testing

* added card section card

* settings form

* homepage cetegory ui

* frontend category placeholder

* fixed broken scheduler

* remove old files

* removed temp test

Co-authored-by: Hayden <hay-kot@pm.me>

* Fix missing translations key (#133)

* translation: add simplified & traditional chinese

* Fix missing translations

* fix chinese translations

* v0.2.0 Release Candidate (#141)

* Fix link to Docker Hub

Found an extra s. DESTROYED it.

* Release v0.1.0 Candidate (#85)

* Changed uvicorn port to 80

* Changed port in docker-compose to match dockerfile

* Readded environment variables in docker-compose

* production image rework

* Use opengraph metadata to make basic recipe cards when full recipe metadata is not available

* fixed instrucitons on parse

* add last_recipe

* automated testing

* roadmap update

* Sqlite (#75)

* file structure

* auto-test

* take 2

* refactor ap scheduler and startup process

* fixed scraper error

* database abstraction

* database abstraction

* port recipes over to new schema

* meal migration

* start settings migration

* finale mongo port

* backup improvements

* migration imports to new DB structure

* unused import cleanup

* docs strings

* settings and theme import logic

* cleanup

* fixed tinydb error

* requirements

* fuzzy search

* remove scratch file

* sqlalchemy models

* improved search ui

* recipe models almost done

* sql modal population

* del scratch

* rewrite database model mixins

* mostly grabage

* recipe updates

* working sqllite

* remove old files and reorganize

* final cleanup

Co-authored-by: Hayden <hay-kot@pm.me>

* Backup card (#78)

* backup / import dialog

* upgrade to new tag method

* New import card

* rename settings.py to app_config.py

* migrate to poetry for development

* fix failing test

Co-authored-by: Hayden <hay-kot@pm.me>

* added mkdocs to docker-compose

* Translations (#72)

* Translations + danish

* changed back proxy target to use ENV

* Resolved more merge conflicts

* Removed test in translation

* Documentation of translations

* Updated translations

* removed old packages

Co-authored-by: Hayden <64056131+hay-kot@users.noreply.github.com>

* fail to start bug fixes

* feature: prep/cook/total time slots (#80)

Co-authored-by: Hayden <hay-kot@pm.me>

* missing bind attributes

* Bug fixes (#81)

* fix: url remains after succesful import

* docs: changelog + update todos

* arm image

* arm compose

* compose updates

* update poetry

* arm support

Co-authored-by: Hayden <hay-kot@pm.me>

* dockerfile hotfix

* dockerfile hotfix

* Version Release Final Touches (#84)

* Remove slim

* bug: opacity issues

* bug: startup failure with no database

* ci/cd on dev branch

* formatting

* v0.1.0 documentation

Co-authored-by: Hayden <hay-kot@pm.me>

* db init hotfix

* bug: fix crash in mongo

* fix mongo bug

* fixed version notifier

* finale changelog

Co-authored-by: kentora <=>
Co-authored-by: Hayden <hay-kot@pm.me>
Co-authored-by: Richard Mitic <richard.h.mitic@gmail.com>
Co-authored-by: kentora <kentora@kentora.dk>

* build container

* webscraper hotfix

* notes hot fix

* bug: mongo updates fail #99

* Fix error message (#101)

* gh funding

* Create Issue Templates (#125)

* Create bug_report.md

* Create config.yml

Included a link to feature requests.

* Update config.yml

Fixed link I had for testing to the actual link

* Update bug_report.md

fix capitalization

* Update .github/ISSUE_TEMPLATE/bug_report.md

Co-authored-by: Stephen Brown II <Stephen.Brown2@gmail.com>

Co-authored-by: Stephen Brown II <Stephen.Brown2@gmail.com>

* merge kentors changes

* refactor/recipe routers

* category/tag database relationship and endpoints

* frontend category management

* update branch todos

* bug/normalize recipe steps html

* remove console.log +  refactor categories

* fix categories database errors

* refactor/ router endpoint

* refactor/ remove old code

* drag and drop ingredients

* general cleanup

* route refactoring

* changelog

* api refactoring + random cleanup

* fixed backwards sort

* Update mkdocs.yml (#137)

Fix warning from Deploy Docs github action

* fixed navigate on enter in search

* refactor/create global css

* added category scroll

* cleanup todos

* debug routes

* docs/new gifs & general updates

* cleanup

* fix list test

Co-authored-by: David Young <davidy@funkypenguin.co.nz>
Co-authored-by: Hayden <hay-kot@pm.me>
Co-authored-by: Richard Mitic <richard.h.mitic@gmail.com>
Co-authored-by: kentora <kentora@kentora.dk>
Co-authored-by: Alexei Pesic <pesic.alexei@gmail.com>
Co-authored-by: Andrew <dpieski@gmail.com>
Co-authored-by: Stephen Brown II <Stephen.Brown2@gmail.com>

* fix build

* fix duplicate editor

* fixed docker mount problem

* python 3.9

* added tasks for non-docker development

* remove old scripts

* dev updates

* fixed no image upload option

* get version from backend

* final docs pass

* .gitignore

* feature/markdown support for description and steps

* package-lock

* rename production task

* category import errors on import

Co-authored-by: kentora <=>
Co-authored-by: Hayden <hay-kot@pm.me>
Co-authored-by: Richard Mitic <richard.h.mitic@gmail.com>
Co-authored-by: kentora <kentora@kentora.dk>
Co-authored-by: David Young <davidy@funkypenguin.co.nz>
Co-authored-by: Bastien <43323819+Batgame@users.noreply.github.com>
Co-authored-by: sephrat <34862846+sephrat@users.noreply.github.com>
Co-authored-by: Nick CJ <17556895+nickcj931@users.noreply.github.com>
Co-authored-by: dekvall <dkvldev@gmail.com>
Co-authored-by: wengtad <wengtad93@gmail.com>
Co-authored-by: Alexei Pesic <pesic.alexei@gmail.com>
Co-authored-by: Andrew <dpieski@gmail.com>
Co-authored-by: Stephen Brown II <Stephen.Brown2@gmail.com>
This commit is contained in:
Hayden
2021-02-08 15:14:02 -09:00
committed by GitHub
parent b3573dc078
commit 808f11da0a
10 changed files with 447 additions and 296 deletions

View File

@@ -1,63 +0,0 @@
from typing import List
from app_config import USE_MONGO, USE_SQL
from db.db_base import BaseDocument
from db.db_setup import USE_MONGO, USE_SQL
from db.mongo.meal_models import MealDocument, MealPlanDocument
from db.sql.db_session import create_session
from db.sql.meal_models import MealPlanModel
class _Meals(BaseDocument):
def __init__(self) -> None:
self.primary_key = "uid"
if USE_SQL:
self.sql_model = MealPlanModel
self.create_session = create_session
self.document = MealPlanDocument
@staticmethod
def _process_meals(meals: List[dict]) -> List[MealDocument]:
"""Turns a list of Meals in dictionary form into a list of
MealDocuments that can be attached to a MealPlanDocument
Args: \n
meals (List[dict]): From a Pydantic Class in meal_services.py \n
Returns:
a List of MealDocuments
"""
meal_docs = []
for meal in meals:
meal_doc = MealDocument(**meal)
meal_docs.append(meal_doc)
return meal_docs
def save_new_mongo(self, plan_data: dict) -> None:
"""Saves a new meal plan into the database
Args: \n
plan_data (dict): From a Pydantic Class in meal_services.py \n
"""
if USE_MONGO:
plan_data["meals"] = _Meals._process_meals(plan_data["meals"])
document = self.document(**plan_data)
document.save()
elif USE_SQL:
pass
def update_mongo(self, uid: str, plan_data: dict) -> dict:
if USE_MONGO:
document = self.document.objects.get(uid=uid)
if document:
new_meals = _Meals._process_meals(plan_data["meals"])
document.update(set__meals=new_meals)
document.save()
elif USE_SQL:
pass

View File

@@ -1,68 +0,0 @@
from app_config import USE_MONGO, USE_SQL
from db.db_base import BaseDocument
from db.mongo.recipe_models import RecipeDocument
from db.sql.db_session import create_session
from db.sql.recipe_models import RecipeModel
class _Recipes(BaseDocument):
def __init__(self) -> None:
self.primary_key = "slug"
if USE_SQL:
self.sql_model = RecipeModel
self.create_session = create_session
else:
self.document = RecipeDocument
def save_new_sql(self, recipe_data: dict):
session = self.create_session()
new_recipe = self.sql_model(**recipe_data)
session.add(new_recipe)
session.commit()
return recipe_data
def update_mongo(self, slug: str, new_data: dict) -> None:
if USE_MONGO:
document = self.document.objects.get(slug=slug)
if document:
document.update(set__name=new_data.get("name"))
document.update(set__description=new_data.get("description"))
document.update(set__image=new_data.get("image"))
document.update(set__recipeYield=new_data.get("recipeYield"))
document.update(set__recipeIngredient=new_data.get("recipeIngredient"))
document.update(
set__recipeInstructions=new_data.get("recipeInstructions")
)
document.update(set__totalTime=new_data.get("totalTime"))
document.update(set__slug=new_data.get("slug"))
document.update(set__categories=new_data.get("categories"))
document.update(set__tags=new_data.get("tags"))
document.update(set__notes=new_data.get("notes"))
document.update(set__orgURL=new_data.get("orgURL"))
document.update(set__rating=new_data.get("rating"))
document.update(set__extras=new_data.get("extras"))
document.save()
return new_data
# elif USE_SQL:
# session, recipe = self._query_one(match_value=slug)
# recipe.update(session=session, **new_data)
# recipe_dict = recipe.dict()
# session.commit()
# session.close()
# return recipe_dict
def update_image(self, slug: str, extension: str) -> None:
if USE_MONGO:
document = self.document.objects.get(slug=slug)
if document:
document.update(set__image=f"{slug}.{extension}")
elif USE_SQL:
pass

View File

@@ -1,44 +0,0 @@
from app_config import USE_MONGO, USE_SQL
from db.db_base import BaseDocument
from db.db_setup import USE_MONGO, USE_SQL
from db.mongo.settings_models import SiteSettingsDocument, WebhooksDocument
from db.sql.db_session import create_session
from db.sql.settings_models import SiteSettingsModel
class _Settings(BaseDocument):
def __init__(self) -> None:
self.primary_key = "name"
if USE_SQL:
self.sql_model = SiteSettingsModel
self.create_session = create_session
self.document = SiteSettingsDocument
def save_new(self, main: dict, webhooks: dict) -> str:
if USE_MONGO:
main["webhooks"] = WebhooksDocument(**webhooks)
new_doc = self.document(**main)
return new_doc.save()
elif USE_SQL:
session = create_session()
new_settings = self.sql_model(main.get("name"), webhooks)
session.add(new_settings)
session.commit()
return new_settings.dict()
def update_mongo(self, name: str, new_data: dict) -> dict:
if USE_MONGO:
document = self.document.objects.get(name=name)
if document:
document.update(set__webhooks=WebhooksDocument(**new_data["webhooks"]))
document.save()
elif USE_SQL:
return

View File

@@ -1,56 +0,0 @@
from app_config import USE_MONGO, USE_SQL
from db.db_base import BaseDocument
from db.db_setup import USE_MONGO, USE_SQL
from db.mongo.settings_models import SiteThemeDocument, ThemeColorsDocument
from db.sql.db_session import create_session
from db.sql.theme_models import SiteThemeModel
class _Themes(BaseDocument):
def __init__(self) -> None:
self.primary_key = "name"
if USE_SQL:
self.sql_model = SiteThemeModel
self.create_session = create_session
else:
self.document = SiteThemeDocument
def save_new(self, theme_data: dict) -> None:
if USE_MONGO:
theme_data["colors"] = ThemeColorsDocument(**theme_data["colors"])
document = self.document(**theme_data)
document.save()
elif USE_SQL:
session = self.create_session()
new_theme = self.sql_model(**theme_data)
session.add(new_theme)
session.commit()
return_data = new_theme.dict()
session.close()
return return_data
def update(self, data: dict) -> dict:
if USE_MONGO:
colors = ThemeColorsDocument(**data["colors"])
theme_document = self.document.objects.get(name=data.get("name"))
if theme_document:
theme_document.update(set__colors=colors)
theme_document.save()
else:
raise Exception("No database entry was found to update")
elif USE_SQL:
session, theme_model = self._query_one(
match_value=data["name"], match_key="name"
)
theme_model.update(**data)
session.commit()
session.close()

View File

@@ -7,6 +7,7 @@ import sqlalchemy.orm as orm
from db.sql.model_base import BaseMixins, SqlAlchemyBase
from slugify import slugify
from sqlalchemy.ext.orderinglist import ordering_list
from sqlalchemy.orm import validates
from utils.logger import logger
@@ -43,20 +44,26 @@ recipes2tags = sa.Table(
class Category(SqlAlchemyBase):
__tablename__ = "categories"
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String, index=True)
slug = sa.Column(sa.String, index=True, unique=True)
name = sa.Column(sa.String, index=True, nullable=False)
slug = sa.Column(sa.String, index=True, unique=True, nullable=False)
recipes = orm.relationship(
"RecipeModel", secondary=recipes2categories, back_populates="categories"
)
@validates("name")
def validate_name(self, key, name):
assert not name == ""
return name
def __init__(self, name) -> None:
self.name = name.strip()
self.slug = slugify(name)
@staticmethod
def create_if_not_exist(session, name: str = None):
test_slug = slugify(name)
try:
result = session.query(Category).filter(Category.name == name.strip()).one()
result = session.query(Category).filter(Category.slug == test_slug).one()
if result:
logger.info("Category exists, associating recipe")
return result
@@ -82,12 +89,17 @@ class Category(SqlAlchemyBase):
class Tag(SqlAlchemyBase):
__tablename__ = "tags"
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String, index=True)
slug = sa.Column(sa.String, index=True, unique=True)
name = sa.Column(sa.String, index=True, nullable=False)
slug = sa.Column(sa.String, index=True, unique=True, nullable=False)
recipes = orm.relationship(
"RecipeModel", secondary=recipes2tags, back_populates="tags"
)
@validates("name")
def validate_name(self, key, name):
assert not name == ""
return name
def to_str(self):
return self.name
@@ -105,8 +117,9 @@ class Tag(SqlAlchemyBase):
@staticmethod
def create_if_not_exist(session, name: str = None):
test_slug = slugify(name)
try:
result = session.query(Tag).filter(Tag.name == name.strip()).first()
result = session.query(Tag).filter(Tag.slug == test_slug).first()
if result:
logger.info("Tag exists, associating recipe")
@@ -169,7 +182,7 @@ class RecipeModel(SqlAlchemyBase, BaseMixins):
id = sa.Column(sa.Integer, primary_key=True)
# General Recipe Properties
name = sa.Column(sa.String)
name = sa.Column(sa.String, nullable=False)
description = sa.Column(sa.String)
image = sa.Column(sa.String)
recipeYield = sa.Column(sa.String)
@@ -205,6 +218,11 @@ class RecipeModel(SqlAlchemyBase, BaseMixins):
orgURL = sa.Column(sa.String)
extras: List[ApiExtras] = orm.relationship("ApiExtras", cascade="all, delete")
@validates("name")
def validate_name(self, key, name):
assert not name == ""
return name
def __init__(
self,
session,

View File

@@ -95,9 +95,24 @@ class ImportDatabase:
del recipe_dict["_id"]
del recipe_dict["dateAdded"]
except:
logger.info("Detected new backup Schema, skipping migration...")
return recipe_dict
pass
# Migration from list to Object Type Data
try:
if "" in recipe_dict["tags"]:
recipe_dict["tags"] = [
tag for tag in recipe_dict["tags"] if not tag == ""
]
except:
pass
try:
if "" in recipe_dict["categories"]:
recipe_dict["categories"] = [
cat for cat in recipe_dict["categories"] if not cat == ""
]
except:
pass
if type(recipe_dict["extras"]) == list:
recipe_dict["extras"] = {}