Repository navigation
Conversation
`do init` runs `reindex_course --active`, a synchronous loop over every active course, which makes every launch slow on large instances (#1197). Add `OPENEDX_INIT_REINDEX_COURSES` (default: true) to skip it, and `OPENEDX_REINDEX_COURSES_SCHEDULE` (default: "") to run it from a k8s CronJob instead. Skipping is opt-in rather than automatic: on-publish indexing only covers blocks edited within REINDEX_AGE, so imported courses only reach the courseware index through `reindex_course`. Also add the `k8s-cronjobs` patch and delete CronJobs on `tutor k8s stop`. Close #1549. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Don't let jobs spawned by a CronJob block `tutor k8s do` commands: they inherit Tutor's labels, so `active_job_names()` treated a long scheduled reindex as an active job and `do init`/`launch` waited on it. - Only list k8s/cronjobs.yml in the kustomization when it has content. - Render the schedule with `tojson` so that quotes can't break the YAML. - Docs: mention the control plane timezone, that cms-job overrides don't apply to the CronJob, and use `tutor local run` in the crontab example (it passes -T without a TTY and doesn't need cms to be running). Fix the k8s-cronjobs patch description. - Keep defaults.yml in alphabetical order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #1549.
We reindex courses on every init. Upstream this is still a synchronous loop over every active course, so on instances with hundreds of courses it adds a long, blocking step to every deploy. In practice, operators of large instances end up removing it and reindexing from external cron jobs instead.
This PR adds two settings:
OPENEDX_INIT_REINDEX_COURSES(default:true): set it tofalseto skip the reindex during CMS init. The default keeps the current behaviour.OPENEDX_REINDEX_COURSES_SCHEDULE(default:""): a cron expression. When set,tutor k8s startcreates acms-reindex-coursesCronJob that runsreindex_course --activeon that schedule. It reuses thecms-jobpod spec and setsconcurrencyPolicy: Forbid.Used together, they move course reindexing out of the deploy path and into a schedule.
I first tried skipping the reindex automatically whenever the Meilisearch course index already had documents, but testing showed that's unsafe. On-publish indexing (
update_search_index) is incremental: it only indexes blocks edited withinREINDEX_AGE(60s). So courses that are imported, or whose index is lost, never get their courseware indexed except throughreindex_course. That's why skipping stays an explicit opt-out, documented with this caveat.Docker Compose has no scheduler, so the CronJob is k8s-only. For `test a host crontab entry instead.
Testing
tutor local do initon a fresh install: the reindex runs as before.tutor local do init --limit cNDEX_COURSESset totrue(reindex runs) and tofalse(the stepis skipped and the skip message is printed).kubectl kustomizeon the rendered env with the schedule unscms-reindex-courses` with the expected schedule, command, labels and namespace).tutor k8s startcreated the CronJob, a job tate job --from=cronjob/cms-reindex-coursescompleted successfully,andtutor k8s stop` deleted it. (pending)python -m unittest discover tests).🤖 Generated with Claude Code