log pkg/imports/queue.go @ 5601:1222b777f51f

age author description
Sat, 06 Aug 2022 02:09:57 +0200 Sascha L. Teichmann Made golint finally happy.
Sat, 12 Feb 2022 18:56:55 +0100 Sascha L. Teichmann Log export: Added marker interface to JobCreators that log messages should be loaded at export.
Fri, 22 Oct 2021 20:11:19 +0200 Sascha L. Teichmann Try harder to retry executing review finalization jobs if they fail: 500 times each 10 min.
Mon, 20 Sep 2021 17:45:39 +0200 Sascha L. Teichmann Use own logging package. logging
Tue, 14 Apr 2020 11:36:13 +0200 Sascha L. Teichmann Import queue: Limit number of repetions of failed jobs to 200 and wait at least 5 seconds between the attempts.
Wed, 01 Apr 2020 15:18:12 +0200 Sascha L. Teichmann Send the info if an import job was re-scheduled to sync callers.
Fri, 27 Mar 2020 00:40:52 +0100 Sascha L. Teichmann Moved misplaced log line around. queued-stage-done
Thu, 26 Mar 2020 22:24:45 +0100 Sascha L. Teichmann Added mechanism to have sync import jobs on import queue. queued-stage-done
Thu, 26 Mar 2020 14:41:23 +0100 Tom Gottfried Prioritize review jobs on selection queued-stage-done
Thu, 26 Mar 2020 13:53:05 +0100 Sascha L. Teichmann Removed unused constant. queued-stage-done
Thu, 26 Mar 2020 13:51:39 +0100 Sascha L. Teichmann Catch another endless recursin call. queued-stage-done
Thu, 26 Mar 2020 13:48:41 +0100 Sascha L. Teichmann Don't write "import #42 started" into imports logs and remove the supress code to prevent this in review jobs. queued-stage-done
Wed, 25 Mar 2020 20:57:20 +0100 Sascha L. Teichmann Enqueue review jobs in front of other 'queued' imports. queued-stage-done
Wed, 25 Mar 2020 18:34:54 +0100 Sascha L. Teichmann Fixed supression of first log in review. queued-stage-done
Wed, 25 Mar 2020 18:02:46 +0100 Sascha L. Teichmann Dont leave old jobs behind if retrying remove jobs. queued-stage-done
Wed, 25 Mar 2020 17:58:13 +0100 Sascha L. Teichmann More neutral messages in retries. queued-stage-done
Wed, 25 Mar 2020 17:50:07 +0100 Sascha L. Teichmann Spell trys as tries. queued-stage-done
Wed, 25 Mar 2020 17:44:26 +0100 Sascha L. Teichmann Supress first log in review job. queued-stage-done
Wed, 25 Mar 2020 16:51:45 +0100 Sascha L. Teichmann Enable jobs to be removed by the import queue. queued-stage-done
Wed, 25 Mar 2020 15:33:11 +0100 Sascha L. Teichmann Redirect logging of review job to original import log. queued-stage-done
Wed, 25 Mar 2020 15:11:04 +0100 Sascha L. Teichmann Define constants for review jobs. queued-stage-done
Tue, 24 Mar 2020 17:41:30 +0100 Sascha L. Teichmann Minimize diff to default. queued-stage-done
Tue, 24 Mar 2020 17:32:58 +0100 Sascha L. Teichmann Nmae json field in reviewed job correctly. queued-stage-done
Tue, 24 Mar 2020 16:53:43 +0100 Sascha L. Teichmann End endless recursion. queued-stage-done
Tue, 24 Mar 2020 16:47:49 +0100 Sascha L. Teichmann Proxy the original job creator directly and not only the dependencies. queued-stage-done
Tue, 24 Mar 2020 16:42:10 +0100 Sascha L. Teichmann Register a kind#rev job factory for any job factory. queued-stage-done
Tue, 24 Mar 2020 15:46:37 +0100 Sascha L. Teichmann First version of a reviewed job. I bet it does not work. queued-stage-done
Fri, 20 Mar 2020 12:28:04 +0100 Sascha L. Teichmann Added feedback support for StageDone.
Fri, 20 Mar 2020 10:54:41 +0100 Sascha L. Teichmann Run review decisions SQLs as reviewer instead of queue user to use the rls policies.
Wed, 18 Mar 2020 18:10:19 +0100 Sascha L. Teichmann Fixed SQL statement.
Wed, 18 Mar 2020 17:52:00 +0100 Sascha L. Teichmann Moved import desision logic to import queue (where it belongs).
Fri, 18 Oct 2019 15:34:34 +0200 Sascha L. Teichmann Added a 'changed' column to the import.imports table.
Mon, 05 Aug 2019 18:21:19 +0200 Tom Gottfried Always try to translate to readable error if import failed
Thu, 25 Jul 2019 16:58:27 +0200 Sascha L. Teichmann Deliver times from import queue in UTC. timezone
Mon, 13 May 2019 09:41:35 +0200 Sascha L. Teichmann Made logging of waht is registered to the scheduler and the import queue more deterministic.
Thu, 09 May 2019 16:09:46 +0200 Sascha L. Teichmann Import queue: Fixed exclusive writers.
Thu, 09 May 2019 15:07:09 +0200 Sascha L. Teichmann Import queue: Implemented multiple reader / one writer strategy when locking ressources.
Thu, 09 May 2019 12:49:53 +0200 Sascha L. Teichmann Import queue: Split locked dependencies in exclusively and multiple uses.
Mon, 01 Apr 2019 10:36:14 +0200 Sascha L. Teichmann Import queue: Extend the idea of changeset 2886:b150e5b37afe to some error cases that might happen before the storing of the after import state.
Wed, 27 Mar 2019 17:00:45 +0100 Sascha L. Teichmann Cosmetics: More compact code.
Wed, 27 Mar 2019 12:12:12 +0100 Sascha L. Teichmann Import queue: As it is important to keep the in-memory and the persistent model in sync try harder to store the final state change if it fails.
Mon, 25 Mar 2019 18:17:18 +0100 Sascha L. Teichmann Import queue: Cleanup up debris after an import in all cases if we are not going to retry it.
Mon, 11 Feb 2019 18:35:01 +0100 Sascha L. Teichmann Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Fri, 08 Feb 2019 15:39:17 +0100 Sascha L. Teichmann Import queue: The job kind parameter when creating a new job from a string was never used.
Thu, 24 Jan 2019 12:56:31 +0100 Tom Gottfried Move import queue to its own database namespace
Wed, 23 Jan 2019 17:58:57 +0100 Sascha L. Teichmann Imports: Made retries and the waiting between the attempts configurable.
Wed, 23 Jan 2019 11:20:14 +0100 Sascha L. Teichmann Imports: Added the new state 'unchanged' which can be issued by the imports to indicate that the database is not modified by the particular imports.
Mon, 14 Jan 2019 17:43:28 +0100 Tom Gottfried Row level security for import jobs
Thu, 10 Jan 2019 18:06:30 +0100 Sascha L. Teichmann More on harmonizing logging.
Thu, 10 Jan 2019 17:12:42 +0100 Sascha L. Teichmann Import queue: Fixed SQL statement.
Thu, 10 Jan 2019 16:19:26 +0100 Sascha L. Teichmann Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Thu, 10 Jan 2019 15:15:31 +0100 Sascha L. Teichmann Import queue: Harmonized logging output.
Wed, 09 Jan 2019 17:28:41 +0100 Sascha L. Teichmann Import queue: removed resolved TODO.
Tue, 08 Jan 2019 17:53:01 +0100 Sascha L. Teichmann Re-scheduled imports: be a bit more tolerant about start time.
Tue, 08 Jan 2019 13:35:44 +0100 Sascha L. Teichmann Imports: Made imports re-runnable if they fail.
Thu, 03 Jan 2019 17:53:18 +0100 Sascha L. Teichmann Imports: Resolved golint issues unrelated to exported symbols commenting.
Fri, 21 Dec 2018 10:33:49 +0100 Sascha L. Teichmann Import queue: Moved email notification stuff to separate file.
Fri, 21 Dec 2018 10:25:15 +0100 Sascha L. Teichmann Import queue: In email notifications generate links to <server>/#/?review=<id> when the state is "pending" and <server>/#/?importlog=<id> otherwise.
Thu, 20 Dec 2018 16:00:50 +0100 Sascha L. Teichmann Import email notification: Generate correct link.
Thu, 20 Dec 2018 15:45:09 +0100 Sascha L. Teichmann Import email notifications: Removed leading new line.
Thu, 20 Dec 2018 15:33:29 +0100 Sascha L. Teichmann Import queue: Implemented email notifications.
Thu, 20 Dec 2018 13:45:32 +0100 Sascha L. Teichmann Import queue: Implemented auto-accept and email sending.
Wed, 19 Dec 2018 15:58:44 +0100 Sascha L. Teichmann Refactored: Moved REST /api/imports/scheduler to /api/imports/config
Tue, 04 Dec 2018 18:54:50 +0100 Sascha L. Teichmann Backend: Documented the internal API of the global import queue.
Tue, 04 Dec 2018 18:20:31 +0100 Sascha L. Teichmann The global import queue already knows which kinds of imports it supports.
Wed, 28 Nov 2018 19:55:29 +0100 Sascha L. Teichmann Backend: Removed some trailing white space in some SQL statements.
Wed, 28 Nov 2018 09:52:34 +0100 Sascha L. Teichmann Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sun, 25 Nov 2018 16:26:41 +0100 Sascha L. Teichmann To make golint happier made context.Context to be the first argument in all calls.
Sun, 25 Nov 2018 15:56:07 +0100 Sascha L. Teichmann To make golint happier made context.Context to be the first argument of auth.RunAs.
Fri, 16 Nov 2018 18:35:09 +0100 Sascha L. Teichmann Implemented the db stuff of the review process. Needs testing.
Fri, 16 Nov 2018 12:15:51 +0100 Sascha L. Teichmann The imort queue has now six states:
Fri, 16 Nov 2018 12:08:46 +0100 Sascha L. Teichmann REST GET call to /imports now has the ability to be filtered by kinds or states.
Wed, 14 Nov 2018 12:23:10 +0100 Sascha L. Teichmann Track successfull imports in a separate table to be able to remove them later.
Fri, 09 Nov 2018 10:57:49 +0100 Sascha L. Teichmann Don't issue new lines at end of log messages when importing.
Thu, 08 Nov 2018 15:50:28 +0100 Sascha L. Teichmann Independent imports in terms of affected tables/dependencies are now run concurrently.
Tue, 23 Oct 2018 18:15:14 +0200 Sascha L. Teichmann Added GNU Affero General Public License.
Tue, 23 Oct 2018 10:14:41 +0200 Sascha L. Teichmann Fixed and harmonized wording in importer queue a bit.
Mon, 22 Oct 2018 18:14:16 +0200 Sascha L. Teichmann Re-enqueue import jobs in state running if the the gemma server starts. persistent-import-queue
Mon, 22 Oct 2018 17:38:07 +0200 Sascha L. Teichmann Do not rely on internal gemma state only. Instead poll every 10 seconds for new import jobs. persistent-import-queue
Mon, 22 Oct 2018 17:16:02 +0200 Sascha L. Teichmann Wait with start of import queue until configuration is fully loaded. persistent-import-queue
Mon, 22 Oct 2018 16:54:34 +0200 Sascha L. Teichmann Persist job queue in database. WIP. persistent-import-queue
Mon, 22 Oct 2018 11:24:25 +0200 Sascha L. Teichmann Run do and cleanup of import jobs in own go routines with crash handler.
Mon, 22 Oct 2018 10:45:17 +0200 Sascha L. Teichmann Decoupled import job creation and job execution with a factory function.
Sat, 20 Oct 2018 19:27:33 +0200 Sascha L. Teichmann Typo fix.
Sat, 20 Oct 2018 19:14:00 +0200 Sascha L. Teichmann In preparation of persisting import jobs logging is done through an interface.
Fri, 19 Oct 2018 18:09:26 +0200 Sascha L. Teichmann Store job id alongside to job in job queue.
Fri, 19 Oct 2018 12:14:53 +0200 Sascha L. Teichmann Finally enqueue sounding result import job to import jobs.
Thu, 18 Oct 2018 18:37:19 +0200 Sascha L. Teichmann Started with endpoint for uploading sounding result and trigger respective import job.
Tue, 16 Oct 2018 14:59:32 +0200 Sascha L. Teichmann Started with import queue.