Kingy Grok Bot usage pilot — November 11, 2026

{
  "run_date": "2026-11-11",
  "product": "Grok Bot web in signed-in Chrome",
  "plan": "The X Subscription Premium",
  "design": "Four sequential runs A B B A in one new Bot and conversation, using identical 12-record synthetic data. A requested detailed row explanation, calculations and next actions. B requested only five result fields, no browsing, and an explicit stopping point.",
  "expected": {
    "valid_unpaid_total": 695,
    "overdue_unpaid_total": 510,
    "overdue_ids": [
      "INV101",
      "INV103",
      "INV107",
      "INV110"
    ],
    "duplicate_ids": [
      "INV103"
    ],
    "invalid_ids": [
      "INV106",
      "INV111"
    ]
  },
  "meter": {
    "before_setup": "0%",
    "after_setup_and_A1": "1%",
    "after_B1": "1%",
    "after_B2_read": "1%",
    "final": "1%",
    "next_reset": "Nov 18, 2026",
    "reported_tokens_after_A1": 141971,
    "reported_tokens_after_B1": 194290,
    "reported_tokens_final": 303906,
    "reported_credit_usage": "$0",
    "credit_balance": "$0.00"
  },
  "limitations": [
    "Bot creation/onboarding occurred between the initial meter capture and A1, so startup usage cannot be separated.",
    "Same conversation and repeated identical inputs permit answer reuse and cached context. This is a small workflow pilot, not an independent performance benchmark.",
    "Browser entry collapsed line breaks in all prompts. Both conditions used the same concatenated data representation; exact submitted prompts are retained.",
    "Detailed condition requested more narrative. Five-field correctness does not establish equivalence for tasks requiring the narrative report.",
    "Dashboard updated with delay and showed whole-number percentages. Unchanged 1% does not mean zero usage.",
    "Reported token counts are account dashboard totals including setup, not a per-task tariff or bill. Cached and new tokens need not carry the same price.",
    "CSV export produced no download within 15 seconds; rendered dashboard event rows were captured instead.",
    "Browser connection was interrupted after B1; no task was resent because of that interruption.",
    "No routines, plan tier, billing settings or usage resets were changed/tested.",
    "No incorrect requested reconciliation field was found in the four observed outputs. No general savings or universal allowance conclusion follows."
  ],
  "runs": [
    {
      "run": "A1",
      "condition": "A",
      "submitted_prompt": "As of 2026-11-11, reconcile this synthetic invoice export. Duplicate IDs count once. Negative or nonnumeric amounts are invalid and excluded. Overdue means unpaid and due strictly before the as-of date. Find the valid unpaid total, overdue unpaid total, overdue invoice IDs, duplicate IDs, and invalid invoice IDs. Do not contact anyone, connect accounts, change settings, create routines, or incur additional paid usage. Produce a detailed report explaining each row, your calculations, and the recommended next actions. Recompute from this export.id,supplier,amount,status,dueINV101,Acorn,120,unpaid,2026-11-01INV102,Birch,80,paid,2026-11-02INV103,Cedar,200,unpaid,2026-11-10INV103,Cedar,200,unpaid,2026-11-10INV104,Delta,50,unpaid,2026-11-11INV105,Elm,75,unpaid,2026-11-20INV106,Fir,-30,unpaid,2026-11-05INV107,Gum,100,unpaid,2026-11-07INV108,Hazel,40,paid,2026-11-01INV109,Iris,60,unpaid,2026-11-12INV110,Juniper,90,unpaid,2026-11-08INV111,Koa,unknown,unpaid,2026-11-03",
      "output_format": "rendered accessibility transcript",
      "output": "- paragraph:\n      - text: Valid unpaid comes to\n      - strong: $695\n      - text: and overdue unpaid to\n      - strong: $510\n      - text: \", as of 2026-11-11. I worked this out from the export you pasted, so there were no outside calls or extra costs.\"\n    - paragraph:\n      - strong: Summary\n    - list:\n      - strong: \"Valid unpaid total:\"\n      - text: $695\n      - strong: \"Overdue unpaid total:\"\n      - text: $510\n      - strong: \"Overdue invoice IDs:\"\n      - text: INV101, INV103, INV107, INV110\n      - strong: \"Duplicate IDs:\"\n      - text: INV103 shows up twice with the same details, so I counted it once.\n      - strong: \"Invalid invoice IDs:\"\n      - text: INV106 has a negative amount (−30) and INV111's amount is \"unknown\", so I left both out.\n    - paragraph:\n      - strong: Row by row\n    - table:\n      - rowgroup:\n        - row \"ID Supplier Amount Status Due Result\":\n          - columnheader \"ID\"\n          - columnheader \"Supplier\"\n          - columnheader \"Amount\"\n          - columnheader \"Status\"\n          - columnheader \"Due\"\n          - columnheader \"Result\"\n      - rowgroup:\n        - row \"INV101 Acorn 120 unpaid 2026-11-01 Valid, unpaid, overdue\":\n          - cell \"INV101\"\n          - cell \"Acorn\"\n          - cell \"120\"\n          - cell \"unpaid\"\n          - cell \"2026-11-01\"\n          - cell \"Valid, unpaid, overdue\":\n            - text: Valid, unpaid,\n            - strong: overdue\n        - row \"INV102 Birch 80 paid 2026-11-02 Valid, paid, so it's not counted\":\n          - cell \"INV102\"\n          - cell \"Birch\"\n          - cell \"80\"\n          - cell \"paid\"\n          - cell \"2026-11-02\"\n          - cell \"Valid, paid, so it's not counted\"\n        - row \"INV103 Cedar 200 unpaid 2026-11-10 Valid, unpaid, overdue (first copy)\":\n          - cell \"INV103\"\n          - cell \"Cedar\"\n          - cell \"200\"\n          - cell \"unpaid\"\n          - cell \"2026-11-10\"\n          - cell \"Valid, unpaid, overdue (first copy)\":\n            - text: Valid, unpaid,\n            - strong: overdue\n            - text: (first copy)\n        - row \"INV103 Cedar 200 unpaid 2026-11-10 Duplicate, left out\":\n          - cell \"INV103\"\n          - cell \"Cedar\"\n          - cell \"200\"\n          - cell \"unpaid\"\n          - cell \"2026-11-10\"\n          - cell \"Duplicate, left out\":\n            - strong: Duplicate\n            - text: \", left out\"\n        - row \"INV104 Delta 50 unpaid 2026-11-11 Valid, unpaid, not overdue because it's due today\":\n          - cell \"INV104\"\n          - cell \"Delta\"\n          - cell \"50\"\n          - cell \"unpaid\"\n          - cell \"2026-11-11\"\n          - cell \"Valid, unpaid, not overdue because it's due today\"\n        - row \"INV105 Elm 75 unpaid 2026-11-20 Valid, unpaid, not due yet\":\n          - cell \"INV105\"\n          - cell \"Elm\"\n          - cell \"75\"\n          - cell \"unpaid\"\n          - cell \"2026-11-20\"\n          - cell \"Valid, unpaid, not due yet\"\n        - row \"INV106 Fir −30 unpaid 2026-11-05 Invalid (negative amount)\":\n          - cell \"INV106\"\n          - cell \"Fir\"\n          - cell \"−30\"\n          - cell \"unpaid\"\n          - cell \"2026-11-05\"\n          - cell \"Invalid (negative amount)\":\n            - strong: Invalid\n            - text: (negative amount)\n        - row \"INV107 Gum 100 unpaid 2026-11-07 Valid, unpaid, overdue\":\n          - cell \"INV107\"\n          - cell \"Gum\"\n          - cell \"100\"\n          - cell \"unpaid\"\n          - cell \"2026-11-07\"\n          - cell \"Valid, unpaid, overdue\":\n            - text: Valid, unpaid,\n            - strong: overdue\n        - row \"INV108 Hazel 40 paid 2026-11-01 Valid, paid, so it's not counted\":\n          - cell \"INV108\"\n          - cell \"Hazel\"\n          - cell \"40\"\n          - cell \"paid\"\n          - cell \"2026-11-01\"\n          - cell \"Valid, paid, so it's not counted\"\n        - row \"INV109 Iris 60 unpaid 2026-11-12 Valid, unpaid, not due yet\":\n          - cell \"INV109\"\n          - cell \"Iris\"\n          - cell \"60\"\n          - cell \"unpaid\"\n          - cell \"2026-11-12\"\n          - cell \"Valid, unpaid, not due yet\"\n        - row \"INV110 Juniper 90 unpaid 2026-11-08 Valid, unpaid, overdue\":\n          - cell \"INV110\"\n          - cell \"Juniper\"\n          - cell \"90\"\n          - cell \"unpaid\"\n          - cell \"2026-11-08\"\n          - cell \"Valid, unpaid, overdue\":\n            - text: Valid, unpaid,\n            - strong: overdue\n        - row \"INV111 Koa unknown unpaid 2026-11-03 Invalid (amount isn't a number)\":\n          - cell \"INV111\"\n          - cell \"Koa\"\n          - cell \"unknown\"\n          - cell \"unpaid\"\n          - cell \"2026-11-03\"\n          - cell \"Invalid (amount isn't a number)\":\n            - strong: Invalid\n            - text: (amount isn't a number)\n    - button \"Copy table\":\n    - button \"Save table\":\n    - paragraph:\n      - strong: Calculations\n    - paragraph: \"Valid unpaid is INV101, INV103 (counted once), INV104, INV105, INV107, INV109 and INV110:\"\n    - math:\n      - generic: \"120\"\n      - generic: +\n      - generic: \"200\"\n      - generic: +\n      - generic: \"50\"\n      - generic: +\n      - generic: \"75\"\n      - generic: +\n      - generic: \"100\"\n      - generic: +\n      - generic: \"60\"\n      - generic: +\n      - generic: \"90\"\n      - generic: =\n      - generic: \"695\"\n    - generic: 120 +\n    - generic: 200 +\n    - generic: 50 +\n    - generic: 75 +\n    - generic: 100 +\n    - generic: 60 +\n    - generic: 90 =\n    - generic: \"695\"\n    - paragraph: \"Overdue unpaid only includes invoices due strictly before 2026-11-11, which are INV101, INV103, INV107 and INV110:\"\n    - math:\n      - generic: \"120\"\n      - generic: +\n      - generic: \"200\"\n      - generic: +\n      - generic: \"100\"\n      - generic: +\n      - generic: \"90\"\n      - generic: =\n      - generic: \"510\"\n    - generic: 120 +\n    - generic: 200 +\n    - generic: 100 +\n    - generic: 90 =\n    - generic: \"510\"\n    - paragraph: The two paid invoices, INV102 ($80) and INV108 ($40), aren't in either total.\n    - paragraph:\n      - strong: Recommended next actions\n    - list:\n      - strong: Follow up on the four overdue invoices.\n      - text: Start with INV103 from Cedar, since it's the largest at $200, then INV101 (Acorn), INV107 (Gum) and INV110 (Juniper).\n      - strong: Remove the extra INV103 row\n      - text: at the source system so it can't be paid or reported twice. Check that it really is one invoice and not two separate ones with the same ID.\n      - strong: Check INV106 (Fir, −30).\n      - text: A negative amount usually means it's a credit note. If so, record it as a credit against Fir's balance instead of as an invoice.\n      - strong: Get the real amount for INV111 (Koa).\n      - text: Once you have it, add it back in. It's due 2026-11-03, so it would count as overdue if it's still unpaid.\n      - strong: Watch INV104 (Delta, $50).\n      - text: It's due today and becomes overdue tomorrow if it isn't paid.\n      - strong: INV109 ($60, due 2026-11-12) and INV105 ($75, due 2026-11-20)\n      - text: are due soon, so keep them on the list for this payment cycle.\n    - paragraph: Should I turn this into a CSV of the cleaned-up rows?\n    - button \"React to message\":\n    - button \"Reply to message\":\n    - button \"More message actions\":",
      "grading": "All five requested totals/ID sets correct; manually checked against deterministic answer key.",
      "score": "5/5"
    },
    {
      "run": "B1",
      "condition": "B",
      "submitted_prompt": "As of 2026-11-11, reconcile this synthetic invoice export. Duplicate IDs count once. Negative or nonnumeric amounts are invalid and excluded. Overdue means unpaid and due strictly before the as-of date. Find the valid unpaid total, overdue unpaid total, overdue invoice IDs, duplicate IDs, and invalid invoice IDs. Do not contact anyone, connect accounts, change settings, create routines, or incur additional paid usage. Use only the supplied data; do not browse. Return exactly five labeled lines: valid_unpaid_total, overdue_unpaid_total, overdue_ids, duplicate_ids, invalid_ids. No narrative or follow-up suggestions. Stop when those five lines are complete. Recompute from this export.id,supplier,amount,status,dueINV101,Acorn,120,unpaid,2026-11-01INV102,Birch,80,paid,2026-11-02INV103,Cedar,200,unpaid,2026-11-10INV103,Cedar,200,unpaid,2026-11-10INV104,Delta,50,unpaid,2026-11-11INV105,Elm,75,unpaid,2026-11-20INV106,Fir,-30,unpaid,2026-11-05INV107,Gum,100,unpaid,2026-11-07INV108,Hazel,40,paid,2026-11-01INV109,Iris,60,unpaid,2026-11-12INV110,Juniper,90,unpaid,2026-11-08INV111,Koa,unknown,unpaid,2026-11-03",
      "output_format": "rendered accessibility transcript",
      "output": "- paragraph: \"valid_unpaid_total: 695 overdue_unpaid_total: 510 overdue_ids: INV101, INV103, INV107, INV110 duplicate_ids: INV103 invalid_ids: INV106, INV111\"\n    - button \"React to message\":\n    - button \"Reply to message\":\n    - button \"More message actions\":",
      "grading": "All five requested totals/ID sets correct; manually checked against deterministic answer key.",
      "score": "5/5"
    },
    {
      "run": "B2",
      "condition": "B",
      "submitted_prompt": "As of 2026-11-11, reconcile this synthetic invoice export. Duplicate IDs count once. Negative or nonnumeric amounts are invalid and excluded. Overdue means unpaid and due strictly before the as-of date. Find the valid unpaid total, overdue unpaid total, overdue invoice IDs, duplicate IDs, and invalid invoice IDs. Do not contact anyone, connect accounts, change settings, create routines, or incur additional paid usage. Use only the supplied data; do not browse. Return exactly five labeled lines: valid_unpaid_total, overdue_unpaid_total, overdue_ids, duplicate_ids, invalid_ids. No narrative or follow-up suggestions. Stop when those five lines are complete. Recompute from this export.id,supplier,amount,status,dueINV101,Acorn,120,unpaid,2026-11-01INV102,Birch,80,paid,2026-11-02INV103,Cedar,200,unpaid,2026-11-10INV103,Cedar,200,unpaid,2026-11-10INV104,Delta,50,unpaid,2026-11-11INV105,Elm,75,unpaid,2026-11-20INV106,Fir,-30,unpaid,2026-11-05INV107,Gum,100,unpaid,2026-11-07INV108,Hazel,40,paid,2026-11-01INV109,Iris,60,unpaid,2026-11-12INV110,Juniper,90,unpaid,2026-11-08INV111,Koa,unknown,unpaid,2026-11-03",
      "output_format": "rendered text",
      "output": "valid_unpaid_total: 695\noverdue_unpaid_total: 510\noverdue_ids: INV101, INV103, INV107, INV110\nduplicate_ids: INV103\ninvalid_ids: INV106, INV111",
      "grading": "All five requested totals/ID sets correct; manually checked against deterministic answer key.",
      "score": "5/5"
    },
    {
      "run": "A2",
      "condition": "A",
      "submitted_prompt": "As of 2026-11-11, reconcile this synthetic invoice export. Duplicate IDs count once. Negative or nonnumeric amounts are invalid and excluded. Overdue means unpaid and due strictly before the as-of date. Find the valid unpaid total, overdue unpaid total, overdue invoice IDs, duplicate IDs, and invalid invoice IDs. Do not contact anyone, connect accounts, change settings, create routines, or incur additional paid usage. Produce a detailed report explaining each row, your calculations, and the recommended next actions. Recompute from this export.id,supplier,amount,status,dueINV101,Acorn,120,unpaid,2026-11-01INV102,Birch,80,paid,2026-11-02INV103,Cedar,200,unpaid,2026-11-10INV103,Cedar,200,unpaid,2026-11-10INV104,Delta,50,unpaid,2026-11-11INV105,Elm,75,unpaid,2026-11-20INV106,Fir,-30,unpaid,2026-11-05INV107,Gum,100,unpaid,2026-11-07INV108,Hazel,40,paid,2026-11-01INV109,Iris,60,unpaid,2026-11-12INV110,Juniper,90,unpaid,2026-11-08INV111,Koa,unknown,unpaid,2026-11-03",
      "output_format": "rendered text",
      "output": "Valid unpaid comes to $695 and overdue unpaid to $510, as of 2026-11-11. I recomputed this from the export you pasted, with no outside calls or extra costs.\n\nSummary\n\nValid unpaid total: $695\nOverdue unpaid total: $510\nOverdue invoice IDs: INV101, INV103, INV107, INV110\nDuplicate IDs: INV103 shows up twice with the same details, so I counted it once.\nInvalid invoice IDs: INV106 has a negative amount (−30) and INV111's amount is \"unknown\", so I left both out.\n\nRow by row\n\nID\tSupplier\tAmount\tStatus\tDue\tResult\nINV101\tAcorn\t120\tunpaid\t2026-11-01\tValid, unpaid, overdue\nINV102\tBirch\t80\tpaid\t2026-11-02\tValid, paid, so it's not counted\nINV103\tCedar\t200\tunpaid\t2026-11-10\tValid, unpaid, overdue (first copy)\nINV103\tCedar\t200\tunpaid\t2026-11-10\tDuplicate, left out\nINV104\tDelta\t50\tunpaid\t2026-11-11\tValid, unpaid, not overdue because it's due today\nINV105\tElm\t75\tunpaid\t2026-11-20\tValid, unpaid, not due yet\nINV106\tFir\t−30\tunpaid\t2026-11-05\tInvalid (negative amount)\nINV107\tGum\t100\tunpaid\t2026-11-07\tValid, unpaid, overdue\nINV108\tHazel\t40\tpaid\t2026-11-01\tValid, paid, so it's not counted\nINV109\tIris\t60\tunpaid\t2026-11-12\tValid, unpaid, not due yet\nINV110\tJuniper\t90\tunpaid\t2026-11-08\tValid, unpaid, overdue\nINV111\tKoa\tunknown\tunpaid\t2026-11-03\tInvalid (amount isn't a number)\n\nCalculations\n\nValid unpaid is INV101, INV103 (counted once), INV104, INV105, INV107, INV109 and INV110:\n\n120\n+\n200\n+\n50\n+\n75\n+\n100\n+\n60\n+\n90\n=\n695\n120+200+50+75+100+60+90=695\n\nOverdue unpaid only includes invoices due strictly before 2026-11-11, which are INV101, INV103, INV107 and INV110:\n\n120\n+\n200\n+\n100\n+\n90\n=\n510\n120+200+100+90=510\n\nThe two paid invoices, INV102 ($80) and INV108 ($40), aren't in either total.\n\nRecommended next actions\n\nFollow up on the four overdue invoices. Start with INV103 from Cedar, since it's the largest at $200, then INV101 (Acorn), INV107 (Gum) and INV110 (Juniper).\nRemove the extra INV103 row at the source system so it can't be paid or reported twice. Check that it really is one invoice and not two separate ones with the same ID.\nCheck INV106 (Fir, −30). A negative amount usually means it's a credit note. If so, record it as a credit against Fir's balance instead of as an invoice.\nGet the real amount for INV111 (Koa). Once you have it, add it back in. It's due 2026-11-03, so it would count as overdue if it's still unpaid.\nWatch INV104 (Delta, $50). It's due today and becomes overdue tomorrow if it isn't paid.\nINV109 ($60, due 2026-11-12) and INV105 ($75, due 2026-11-20) are due soon, so keep them on the list for this payment cycle.",
      "grading": "All five requested totals/ID sets correct; manually checked against deterministic answer key.",
      "score": "5/5"
    }
  ]
}

Rendered usage events, not a downloaded CSV
Date (UTC)
	
Product
	
Usage type
	
Model
	
Tokens
	
Cost

Nov 11 at 9:04 PM	Grok Bot	Included	
-
	2.4K	-
Nov 11 at 9:04 PM	Grok Bot	Included	
-
	53.9K	-
Nov 11 at 9:03 PM	Grok Bot	Included	
-
	1.5K	-
Nov 11 at 9:03 PM	Grok Bot	Included	
-
	51.8K	-
Nov 11 at 8:59 PM	Grok Bot	Included	
-
	1.6K	-
Nov 11 at 8:59 PM	Grok Bot	Included	
-
	50.7K	-
Nov 11 at 8:58 PM	Grok Bot	Included	
-
	49.5K	-
Nov 11 at 8:57 PM	Grok Bot	Included	
-
	2.4K	-
Nov 11 at 8:57 PM	Grok Bot	Included	
-
	47.8K	-
Nov 11 at 8:56 PM	Grok Bot	Included	
-
	42.2K	-