Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/backend
Commit message (Collapse)AuthorAgeFilesLines
* sync: fix broken sync_packagesGilles Dartiguelongue2017-01-221-1/+6
| | | | | I think there is a problem in the logic here but at least this gets me past the initial sync.
* frontend: move setup to a separate view classMart Raudsepp2016-12-191-0/+1
|
* models: Add preliminary model and fields for keyword and p.mask storageMart Raudsepp2016-12-071-2/+22
|
* sync: Always handle e-mails in lower case to not end up with duplicatesMart Raudsepp2016-12-072-10/+15
| | | | Suggested-by: Doug Freed <dwfreed@mtu.edu>
* sync: Increase the sync delta to 1 hour and print the sync count and oldest ↵Mart Raudsepp2016-12-071-2/+5
| | | | TS at start
* sync: Don't forget to commit db transaction after all packages are syncedMart Raudsepp2016-12-071-0/+2
| | | | | Sometimes don't need to cancel out, so save the updates after the last modulo 100 to DB too :)
* sync: Order package details syncing based on how old the last sync wasMart Raudsepp2016-12-071-1/+1
| | | | | | This way if we got stuck and re-run much later (or it has exceeded the buffer time constant), we'll at least sync the oldest ones first, so we always end up being less out of date with the oldest sync ts.
* sync: Add package description and maintainers syncMart Raudsepp2016-12-071-4/+45
| | | | | | | | Maintains a sync timestamp to skip recently synced packages, so if a previous run got stuck, we can skip re-doing it too soon. Saves the DB transaction after every 100 packages, because packages.g.o seems to rate-limit us, so at least we will have things saved into DB periodically to cancel out when we get stuck and restart.
* models: Add package maintainers relationship table and ORM relationshipsMart Raudsepp2016-12-071-0/+9
|
* models: Add description and last_sync_ts columns for PackageMart Raudsepp2016-12-071-0/+4
| | | | | | | | | | | | | | | | description we will get from package.g.o per-package detailed json, last_sync_ts will be used to record when that detailed json was last pulled, so that we can rate-limit as-needed. If still using sqlite, can DROP TABLE package; and re-create with ./manage.py init or add the columns manually ALTER TABLE package ADD COLUMN description VARCHAR(500); ALTER TABLE package ADD COLUMN last_sync_ts TIMESTAMP NOT NULL; though that NOT NULL vs default on sqlalchemy's side for now might pose an issue, solving of which is an easy exercise for those that care instead of recreating.
* frontend: Setup Flask-WTF and use it for following maintainer checkboxes displayMart Raudsepp2016-12-051-0/+1
| | | | No POST handling yet.
* sync: return empty dict on projects retrieval error, so the caller won't errorMart Raudsepp2016-12-051-3/+3
|
* sync: use requests response.json() directly instead of json.loadsMart Raudsepp2016-12-041-4/+3
| | | | | | This should ensure requests will handle UTF-8 fully correctly for us Suggested-by: Doug Freed <dwfreed@mtu.edu>
* sync: Fix UTF-8 handling for projects.xml importMart Raudsepp2016-12-041-1/+1
| | | | | | | | | Need to feed response.content bytestring into ElementTree, not response.text. With the latter ET seems to figure it's already decoded and goes all latin-1 on us. From response.content bytestream it notices the UTF-8 encoding XML markup and does things right. Diagnosed-by: Doug Freed <dwfreed@mtu.edu>
* sync: Update individual maintainer names during projects sync for the time beingMart Raudsepp2016-12-041-2/+4
| | | | | ... until we don't have master data for this that we shouldn't overwrite. Also remove a now done TODO item and tweak a debug string I messed up pre-commit.
* sync: Project members and subprojects syncing to DBMart Raudsepp2016-12-041-1/+25
|
* models: Add association table and ORM relationship between projects and membersMart Raudsepp2016-12-041-0/+12
|
* sync: Initial projects syncing to DB without membersMart Raudsepp2016-12-041-7/+28
|
* models: Add Maintainer modelMart Raudsepp2016-12-041-0/+11
| | | | | As this is a new table, just re-doing "./manage.py init" should add it to db, while keeping old data.
* models: Use sqlalchemy Unicode columns instead of StringMart Raudsepp2016-12-041-4/+4
|
* Turn off flask-sqlalchemy modification tracking to silence a warning and ↵Mart Raudsepp2016-11-281-0/+1
| | | | | | | | | | save resources We shouldn't need SQLALCHEMY_TRACK_MODIFICATIONS, so just disable it as suggested to avoid the warning and save these little resources. This is a flask-sqlalchemy specific session event tracking thing; sqlalchemy itself provides its own mechanism for this these days, and if we need something like that, we should probably use those, not this flask-sqlalchemy thing.
* Make the dummy initial web frontend prettyMart Raudsepp2016-11-231-12/+4
| | | | | | | | | | | Now uses extra Flask-Classy dependency for nicer routing and organization. Stylesheet is Gentoo Tyrian loaded from standard CDN location. Reorganizes some of frontend to frontend module, which necessitates telling Flask it's "frontend", not "backend" (for templates to work without passing custom paths). If reorganization goes this route and completes, all the flask parts should end up in frontend module, making this hack obsolete. Though backend might want to use Flask-Sqlalchemy too, so needing the Flask app object, but not sure yet.
* Add parsed project members to the result dictMart Raudsepp2016-11-111-0/+3
|
* Normalize subproject inherit-members to True or False during parsingMart Raudsepp2016-11-101-2/+2
|
* Initial projects.xml parsing code with debug printoutMart Raudsepp2016-11-101-4/+55
|
* Add dirty sync_versions debug codeMart Raudsepp2016-11-101-1/+11
| | | | | | | This just prints the first packages versions JSON data out and exits, so just some initial debug code out of the way to sync in projects.xml first, as sync_versions will need to reference projects and maintainers, so better to finish projects.xml sync first.
* Add a full_name property to package and remove some debug spam on syncMart Raudsepp2016-09-242-1/+4
|
* Add syncing of packages in categories from packages.g.o (just name)Mart Raudsepp2016-09-071-2/+26
| | | | Also add manage.py commands to call the sync steps individually for testing
* Don't double-quote debug output for full atoms from %r usageMart Raudsepp2016-09-071-2/+2
|
* Add initial code to sync categories from packages.g.o with associated plumbingMart Raudsepp2016-09-073-1/+63
| | | | | | | Now this should make http://localhost:5000 show the available categories: ./manage.py init ./manage.py sync_gentoo ./manage.py runserver
* Initial start of new Project Grumpy code; Hello World!Mart Raudsepp2016-09-061-0/+7
virtualenv -p python3 venv venv/bin/activate pip install -r requirements.txt ./manage.py runserver --help ./manage.py shell for a python shell with flask instance exported into "app" var ./manage.py runserver