Repository navigation
gmail: an operator inside a quoted run is phrase text, not the operator - #1446
aryankalantri wants to merge 1 commit into
Conversation
…rator's lone quote as its value
nuriyoo
left a comment
There was a problem hiding this comment.
Thanks for this. I measured the spellings around the two bracket rules against gmail.googleapis.com on 2026-11-21, with a Workspace member's gmail.readonly token on its own mailbox of 840 messages, comparing messages.list id sets between 19:31 and 19:36 UTC. On this head K:"(from:me and K:"(from:me) answer nothing for all seven operators, as measured. Three things differ from the measurements, and the first is a row of the issue:
- A quoted
subject:value with a closing bracket (inline, in_take). The issue's 2026-11-10 row hassubject:"(probe)"answering whatsubject:probeanswers; on this head it searches subjects forprobe)and answers nothing. On 2026-11-21subject:"probe)",subject:"( probe",subject:"(probe )",subject:"[probe]"andsubject:"probe]"answered whatsubject:probeanswers too, and the same held forlabel:andin::label:"(inbox)",label:"(inbox",label:"[inbox]",label:"( inbox )",label:"((inbox))",in:"(inbox)"andin:"(sent)"answered whatlabel:inboxandin:sentanswer. The suggestion drops brackets and spaces at both ends of a quotedsubject:,label:orin:value. The address operators already read through them:from:"(me)"andto:"(me)"answered whatfrom:meandto:meanswer, and this head agrees. label:"(inbox)with a lone quote (inline, in_gmail_group_items). Real answered whatlabel:inboxanswers, and-label:"(inbox)its negation.mainanswerslabel:"(inbox)that way; this head answers nothing, since the glued bracket now stays in the label's value. Item 1's suggestion covers it: the lone quote takes the rest of the search as the value,(inbox), and the brackets at its ends drop. The suggestion on the comment names the 2026-11-21 rows for the other six operators.- A lone quote glued to a bracket that no
)closes (body diff,_GMAIL_LONE_QUOTE_UNCLOSED). Real answered nothing forsubject:"(probe,subject:"((probe),label:"(inbox,in:"(inbox,from:"(me,to:"(me,cc:"(me,bcc:"(meandfrom:me subject:"(probe, and every message for-subject:"(probe,-label:"(inboxandNOT subject:"(probe: the rule_GMAIL_LONE_QUOTE_UNCLOSEDrecords with whitespace after the quote holds without it. On this headsubject:"(probeanswers whatsubject:probeanswers,from:"(bobwhatfrom:bobanswers, and-subject:"(probewhat-subject:probeanswers. The diff below lets the whitespace before the bracket be empty; the pattern is outside this PR's lines, so the change is here rather than inline. With itsubject:"(from:mereaches that rule before_gmail_group_items.
Smaller:
- "still reads as" in the
_takecomment and "as before" in the_gmail_group_itemscomment describe the change rather than the rule (inline). - The two test comments restate the code comments; the suggestions point at them instead. The over-the-wire test checks
subject:"(probe"againstsubject:probe, which is empty in that corpus, so both sides are empty onmainas well; the suggestion usesrollout, whichRollout planholds, and adds the label and open-bracket rows (inline).
Nit: the short lines at backlot/routers/google.py:1687-1688 and tests/test_search.py:801 are inside the suggestions, which rewrap them.
With the suggestions and the diff applied, tests/test_search.py and tests/test_google.py pass, and the new rows fail on this head. If you take them, the description's "Why" can name the 2026-11-10 and 2026-11-21 rows.
--- a/backlot/routers/google.py
+++ b/backlot/routers/google.py
@@ -852,16 +852,20 @@
r'^[ \t]*"[ \t]+-*(?=(?i:from|to|cc|bcc|subject|label|in|has|after|before|newer|older'
r"|newer_than|older_than):\S+[ \t]*$)"
)
-# A lone quote after `subject:`, `label:`, `in:` or an address operator, then whitespace and a
-# `(` that no `)` closes. On 2026-10-25 (the record at `_GMAIL_NOT`): `subject:" (from:me`,
-# `subject:" ( from:me`, `subject:" (probe`, `subject:" ((from:me)`, `subject:" (from:me) (to:me`,
-# `label:" (from:me`, `from:" (from:me`, `to:" (from:me` and `probe subject:" (from:me` answered
-# nothing, and `-subject:" (from:me`, `-subject:" (probe`, `-label:" (from:me` and
-# `NOT subject:" (from:me` every message, where `subject:" (from:me)`, with the `)`, answered the
-# ids of `from:me`. `_parse_gmail_q` reads it before `_gmail_groups` can read the group.
+# A lone quote after `subject:`, `label:`, `in:` or an address operator, then a `(` that no `)`
+# closes, with or without whitespace in between. On 2026-10-25 (the record at `_GMAIL_NOT`):
+# `subject:" (from:me`, `subject:" ( from:me`, `subject:" (probe`, `subject:" ((from:me)`,
+# `subject:" (from:me) (to:me`, `label:" (from:me`, `from:" (from:me`, `to:" (from:me`,
+# `probe subject:" (from:me` and `subject:"(from:me` answered nothing, and `-subject:" (from:me`,
+# `-subject:" (probe`, `-label:" (from:me` and `NOT subject:" (from:me` every message, where
+# `subject:" (from:me)`, with the `)`, answered the ids of `from:me`. On 2026-11-21
+# `subject:"(probe`, `subject:"((probe)`, `label:"(inbox`, `in:"(inbox`, `from:"(me`, `to:"(me`,
+# `cc:"(me`, `bcc:"(me`, `label:"(from:me` and `from:me subject:"(probe` answered nothing, and
+# `-subject:"(probe`, `-label:"(inbox` and `NOT subject:"(probe` every message. `_parse_gmail_q`
+# reads it before `_gmail_groups` can read the group.
_GMAIL_LONE_QUOTE_UNCLOSED = re.compile(
r'(?<!\S)((?:NOT\s+)*)(-*)(?:subject|label|in|from|to|cc|bcc):\s*(?:"[ \t]*"\s*)*'
- r'"(?![^"]*")\s+(.*)',
+ r'"(?![^"]*")\s*(.*)',
re.IGNORECASE,
)
# A quoted run holding one operator and its value, with nothing around them but whitespace,| # A bracket glued to an operator's lone quote is no group: the quote belongs to | ||
| # the operator, and what follows it is the operator's value, brackets and all. | ||
| # Measured against gmail.googleapis.com on 2026-10-25: | ||
| # `subject:"(from:me` and `subject:"(from:me)` answered nothing, where | ||
| # `subject:" (from:me)`, with a space after the quote, answered what `from:me` | ||
| # answers. A bracket after a space still opens a group, as before. |
There was a problem hiding this comment.
Item 2: the comment names the 2026-11-21 rows for all seven operators and drops "as before". With the body diff applied, a bracket left open never reaches this branch, so the rows here are the closed ones.
| # A bracket glued to an operator's lone quote is no group: the quote belongs to | |
| # the operator, and what follows it is the operator's value, brackets and all. | |
| # Measured against gmail.googleapis.com on 2026-10-25: | |
| # `subject:"(from:me` and `subject:"(from:me)` answered nothing, where | |
| # `subject:" (from:me)`, with a space after the quote, answered what `from:me` | |
| # answers. A bracket after a space still opens a group, as before. | |
| # A bracket glued to an operator's lone quote is no group: the quote takes the rest | |
| # of the search as the operator's value, brackets and all (`_GMAIL_LONE_QUOTE`). | |
| # Measured against gmail.googleapis.com: on 2026-10-25 `subject:"(from:me)` | |
| # answered nothing, where `subject:" (from:me)`, with a space after the quote, | |
| # answered what `from:me` answers; on 2026-11-21 `label:"(from:me)`, | |
| # `in:"(from:me)`, `from:"(from:me)`, `to:"(from:me)`, `cc:"(from:me)`, | |
| # `bcc:"(from:me)` and `subject:"(probe) from:me` answered nothing, and | |
| # `subject:"(probe)` and `label:"(inbox)` what `subject:probe` and `label:inbox` | |
| # answer. |
| # still reads as `label:inbox`. The closing quote is looked for after the | ||
| # operator's colon, not after the match: the value itself can swallow it. |
There was a problem hiding this comment.
Item 4: "still" describes the change; the rule reads the same without it.
| # still reads as `label:inbox`. The closing quote is looked for after the | |
| # operator's colon, not after the match: the value itself can swallow it. | |
| # reads as `label:inbox`. The closing quote is looked for after the operator's | |
| # colon, not after the match: the value itself can swallow it. |
| # A quoted `subject:` value starting with a bracket reads as the word: the bracket | ||
| # is punctuation around the value, not part of it. Measured against | ||
| # gmail.googleapis.com on 2026-10-25: `subject:"(probe"` answered | ||
| # what `subject:probe` answers. | ||
| if key == "subject" and value.startswith('"'): | ||
| value = '"' + value[1:].lstrip("(") |
There was a problem hiding this comment.
Item 1: brackets and spaces drop at both ends of a quoted subject:, label: or in: value, as measured on 2026-10-25, 2026-11-10 and 2026-11-21.
| # A quoted `subject:` value starting with a bracket reads as the word: the bracket | |
| # is punctuation around the value, not part of it. Measured against | |
| # gmail.googleapis.com on 2026-10-25: `subject:"(probe"` answered | |
| # what `subject:probe` answers. | |
| if key == "subject" and value.startswith('"'): | |
| value = '"' + value[1:].lstrip("(") | |
| # Brackets and spaces at the ends of a quoted `subject:`, `label:` or `in:` value are | |
| # no part of it. Measured against gmail.googleapis.com: `subject:"(probe"` answered | |
| # what `subject:probe` answers on 2026-10-25, and `subject:"(probe)"` on 2026-11-10; | |
| # on 2026-11-21 `subject:"((probe"`, `subject:"probe)"`, `subject:"( probe"`, | |
| # `subject:"(probe )"`, `subject:"[probe]"` and `subject:"probe]"` did too, and | |
| # `label:"(inbox)"`, `label:"(inbox"`, `label:"[inbox]"`, `label:"( inbox )"`, | |
| # `label:"((inbox))"`, `in:"(inbox)"` and `in:"(sent)"` answered what `label:inbox` | |
| # and `in:sent` answer. | |
| if key in _GMAIL_NEGATABLE and value.startswith('"'): | |
| value = '"' + value.strip('"').strip(" ()[]") + '"' |
| # A quoted run holding an operator and another term is a phrase, not the | ||
| # operator; a bracket glued to an operator's lone quote is no group, so the value | ||
| # keeps its bracket; and a quoted `subject:` value starting with a bracket reads as | ||
| # the word. A quoted run holding the operator alone still reads as that operator. |
There was a problem hiding this comment.
Item 5: a pointer to the comments that hold the measurements.
| # A quoted run holding an operator and another term is a phrase, not the | |
| # operator; a bracket glued to an operator's lone quote is no group, so the value | |
| # keeps its bracket; and a quoted `subject:` value starting with a bracket reads as | |
| # the word. A quoted run holding the operator alone still reads as that operator. | |
| # The measurements in `_parse_gmail_q`'s `_take`, at `_GMAIL_LONE_QUOTE_UNCLOSED` and in | |
| # `_gmail_group_items`. |
| ('subject:"(from:me', ("", {"subject": ["from:me"]})), | ||
| ('subject:"(from:me)', ("", {"subject": ["from:me"]})), | ||
| ('subject:"(probe"', ("", {"subject": ["probe"]})), |
There was a problem hiding this comment.
Items 1 and 3: subject:"(from:me reads at _GMAIL_LONE_QUOTE_UNCLOSED with the body diff, and rows for a closing bracket, […], label: and the negated open bracket.
| ('subject:"(from:me', ("", {"subject": ["from:me"]})), | |
| ('subject:"(from:me)', ("", {"subject": ["from:me"]})), | |
| ('subject:"(probe"', ("", {"subject": ["probe"]})), | |
| ('subject:"(from:me', ("", {"": ['subject:"(from:me']})), | |
| ('-subject:"(probe', ("", {})), | |
| ('subject:"(from:me)', ("", {"subject": ["from:me"]})), | |
| ('subject:"(probe"', ("", {"subject": ["probe"]})), | |
| ('subject:"(probe)"', ("", {"subject": ["probe"]})), | |
| ('subject:"[probe]"', ("", {"subject": ["probe"]})), | |
| ('label:"(inbox)', ("", {"label": ["inbox"]})), |
| # An operator inside a quoted run is no operator, so the run is a phrase | ||
| # that matches nothing here; a bracket glued to an operator's lone quote is no group. | ||
| for q in ( | ||
| '"probe from:ava"', | ||
| '"NOT from:ava"', | ||
| '"(from:ava) probe"', | ||
| 'subject:"(from:ava', | ||
| 'subject:"(from:ava)', | ||
| ): | ||
| assert ids(q) == set(), q | ||
| assert ids('""') == ids('"()"') == every | ||
| assert ids('subject:"(probe"') == ids("subject:probe") == set() |
There was a problem hiding this comment.
Item 5: subject:rollout is not empty in this corpus, so the bracket rows tell the two readings apart; the label and open-bracket rows run over the wire too.
| # An operator inside a quoted run is no operator, so the run is a phrase | |
| # that matches nothing here; a bracket glued to an operator's lone quote is no group. | |
| for q in ( | |
| '"probe from:ava"', | |
| '"NOT from:ava"', | |
| '"(from:ava) probe"', | |
| 'subject:"(from:ava', | |
| 'subject:"(from:ava)', | |
| ): | |
| assert ids(q) == set(), q | |
| assert ids('""') == ids('"()"') == every | |
| assert ids('subject:"(probe"') == ids("subject:probe") == set() | |
| for q in ( | |
| '"probe from:ava"', | |
| '"NOT from:ava"', | |
| '"(from:ava) probe"', | |
| 'subject:"(from:ava', | |
| 'subject:"(from:ava)', | |
| 'subject:"(rollout', | |
| 'from:"(bob', | |
| 'subject:"(rollout) from:bob', | |
| ): | |
| assert ids(q) == set(), q | |
| assert ids('""') == ids('"()"') == ids('-subject:"(rollout') == every | |
| rollout = ids("subject:rollout") | |
| assert rollout and rollout != every | |
| for q in ( | |
| 'subject:"(rollout"', | |
| 'subject:"(rollout)"', | |
| 'subject:"[rollout]"', | |
| 'subject:"(rollout)', | |
| ): | |
| assert ids(q) == rollout, q | |
| assert ids('label:"(inbox)"') == ids('label:"(inbox)') == ids("label:inbox") | |
| assert ids('-label:"(inbox)') == every - ids("label:inbox") |
What changed
Backlot now reads a Gmail query the way Gmail does when a quoted run or a bracket touches an operator. An operator inside a quoted run is no operator, so the run is the phrase; a bracket glued to an operator's lone quote opens no group, so what follows the quote is the operator's value; and a quoted
subject:value starting with a bracket reads as the word, the bracket being punctuation around the value, not part of it.Why this is what the real API does
The table was measured against gmail.googleapis.com on 2026-10-25:
"probe from:me","from:me probe","NOT from:me","(NOT from:me)","(from:me) probe"and"(probe from:me)"answered nothing, where a quoted run holding the operator alone still reads as that operator;subject:"(from:meandsubject:"(from:me)answered nothing, wheresubject:" (from:me)answered whatfrom:meanswers;subject:"(probe"answered whatsubject:probeanswers.""and"()"already answered every message on main, so that row needed no code change.Closes #1030
Verification
Tests —
pytest tests/test_search.py tests/test_google.py -qon a.[dev]install assembled offline from wheels fetched via the PyPI JSON API (no index access): 3146 passed, 15 failed, 235 errors. The 15 failures and 235 errors are pre-existing environment gaps in this offline venv, none of them gmail: tests that call httpx directly fail with httpx InvalidURL, and other surfaces error at fixture setup on missing optional dependencies. A sample of the failing tests fails identically on pristine main. Every gmail test passes, and the issue's 9-query repro over the wire on the issue's corpus is all OK.Optional surfaces that ran rather than skipped — the Gmail surface only (
tests/test_search.py,tests/test_google.py); other surfaces were not run, their optional dependencies are not installed in this offline venv.Lint —
ruff check .all checks passed;ruff format --check .206 files already formatted;python scripts/prose_check.pyno findings.A test fails without this change
A new endpoint serving corpus content is ACL-scoped, proved for both an admin and a scoped token
Comments state what was measured, not the history of the fix