Skip to content

Improve test coverage with Vitest #71

Open
Open

Description

@Brayden

Is your feature request related to a problem? Please describe.
Currently the test coverage of the project is nearly non-existent. We do have and use Vitest as a dependency in the Starbase project already that future tests should also use. The idea is to begin accumulating test coverage throughout the entire project.

Goal is to bring test coverage up to above 75% with meaningful tests.

Describe the solution you'd like
Important: Tests should not just be made for line coverage, they should also include meaningful tests that help reveal when issues arise. For example, if we were to write a test for our LiteREST code to see if making a request fetched data we would pass it potentially:

  • a real table name
  • a fake table name
  • no table name (omitted altogether)
  • -1 or null as a value

When PR's are made we run the following Github test action (https://github.com/outerbase/starbasedb/blob/main/.github/workflows/test.yaml) that verifies the project can execute a pnpm i and then run tests with pnpm test. All tests added should be executed during that same pnpm test command.

Describe alternatives you've considered
N/A

Additional context
N/A

Activity

  1. Brayden commented on Jan 24, 2025

    @Brayden
    MemberAuthor

    /bounty $250

  2. algora-pbc commented on Jan 24, 2025

    @algora-pbc

    💎 $250 bounty • Outerbase (YC W23)

    Steps to solve:

    1. Start working: Comment /attempt #71 with your implementation plan
    2. Submit work: Create a pull request including /claim #71 in the PR body to claim the bounty
    3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

    ❗ Important guidelines:

    • To claim a bounty, you need to provide a short demo video of your changes in your pull request
    • If anything is unclear, ask for clarification before starting as this will help avoid potential rework
    • Low quality AI PRs will not receive review and will be closed
    • Do not ask to be assigned unless you've contributed before

    Thank you for contributing to outerbase/starbasedb!

    Attempt Started (UTC) Solution Actions
    🟢 @AlonePenguin Jun 01, 2026, 01:07:27 PM #263 Reward
    🟢 @Rafly0078 Jun 03, 2026, 03:26:57 PM #268 Reward
    🟢 @TechEnthusGH Jun 03, 2026, 06:06:14 PM #269 Reward
    🟢 @tunguz May 11, 2026, 10:11:28 PM #149 Reward
    🟢 @INDICUMA May 11, 2026, 11:47:29 PM #150 Reward
    🟢 @Mykidzou May 12, 2026, 05:44:09 AM #151 Reward
    🟢 @Iineman2 May 12, 2026, 06:54:13 AM #153 Reward
    🟢 @joshpi-1345 May 12, 2026, 07:13:11 AM #156 Reward
    🟢 @Gimyoonsoo May 12, 2026, 07:09:44 AM #155 Reward
    🟢 @ScryUser May 12, 2026, 07:41:20 AM #159 Reward
    🟢 @hieunt170746 May 12, 2026, 07:45:19 AM #157 Reward
    🟢 @kanjk0olb0y-a11y May 12, 2026, 07:09:07 AM #154 Reward
    🟢 @kaziiza May 12, 2026, 09:52:18 AM #163 Reward
    🟢 @kingzzoov-ctrl May 12, 2026, 09:19:20 AM #162 Reward
    🟢 @akamabu May 12, 2026, 09:13:13 AM #161 Reward
    🟢 @obmakesomething May 12, 2026, 10:09:53 AM #164 Reward
    🟢 @thedalbee May 12, 2026, 02:46:57 PM #165 Reward
    🟢 @shengtenghou4-star May 12, 2026, 02:55:17 PM #166 Reward
    🟢 @xingxi0614-cpu May 12, 2026, 04:25:06 PM #169 Reward
    🟢 @mt211211 May 12, 2026, 06:58:24 PM #170 Reward
    🟢 @KaannKara May 12, 2026, 07:23:16 PM #171 Reward
    🟢 @Treasure520520 May 12, 2026, 07:57:13 PM #176 Reward
    🟢 @popsiclelmlm May 12, 2026, 08:35:40 PM #185 Reward
    🟢 @newmattock May 12, 2026, 08:23:40 PM #182 Reward
    🟢 @treydoe1 May 12, 2026, 09:18:48 PM #186 Reward
    🟢 @TheWaste11202 May 12, 2026, 10:48:04 PM #187 Reward
    🟢 @jorgeschulz May 12, 2026, 10:33:47 PM #189 Reward
    🟢 @lisiqi1983 May 13, 2026, 02:45:22 AM #199 Reward
    🟢 @JetMatrix May 13, 2026, 02:13:31 AM #197 Reward
    🟢 @benbenlijie May 13, 2026, 03:25:39 AM #200 Reward
    🟢 @taherdhanera May 13, 2026, 09:13:41 AM #204 Reward
    🟢 @richboyneedcash May 13, 2026, 03:30:05 PM #207 Reward
    🟢 @imaanmulji May 13, 2026, 05:53:26 PM #209 Reward
    🟢 @himanalot May 13, 2026, 06:19:32 PM #210 Reward
    🟢 @duct-tape2 May 14, 2026, 09:15:59 AM #211 Reward
    🟢 @sureshchouksey8 May 14, 2026, 10:55:02 AM #214 Reward
    🟢 @RupteRyuxh May 14, 2026, 08:37:31 PM WIP
    🟢 @singhaditya21 Jun 15, 2026, 05:57:30 AM #277 Reward
    🟢 @oocheol May 18, 2026, 04:10:28 AM #226 Reward
    🟢 @manuelsampedro1 May 18, 2026, 10:18:32 PM #227 Reward
    🟢 @mkcash May 19, 2026, 09:20:28 AM WIP
    🟢 @Qian001A May 22, 2026, 11:47:30 AM #242 Reward
    🟢 @gebibd00-jpg May 22, 2026, 11:03:09 AM #241 Reward
    🟢 @Myc911 Apr 23, 2026, 02:48:29 AM WIP
    🟢 @dominyyin-sudo May 23, 2026, 05:50:38 AM #247 Reward
    🟢 @pofuhermes-bit Apr 23, 2026, 06:45:21 AM WIP
    🟢 @varshith257 Jan 24, 2025, 04:26:08 PM #74 Reward
    🟢 @aybanda Jan 24, 2025, 06:00:30 PM WIP
    🟢 @adliebe May 27, 2026, 09:11:55 AM #252 Reward
    🟢 @jasperan May 27, 2026, 11:39:15 AM #253 Reward
    🟢 @kpassito May 27, 2026, 06:37:55 PM #254 Reward
    🟢 @Entr0zy May 30, 2026, 11:45:23 AM #257 Reward
    🟢 @kodedihp May 30, 2026, 01:00:31 PM WIP
    🟢 @attaboy11 May 30, 2026, 10:36:10 PM #261 Reward
  3. varshith257 commented on Jan 24, 2025

    @varshith257
    Contributor

    /attempt #71

    Algora profile Completed bounties Tech Active attempts Options
    @varshith257 25 bounties from 11 projects
    Scala, Rust,
    TypeScript & more
    Cancel attempt
  4. aybanda commented on Jan 24, 2025

    @aybanda

    /attempt #71

    Algora profile Completed bounties Tech Active attempts Options
    @aybanda 2 bounties from 2 projects
    TypeScript, Python,
    Julia & more
    Cancel attempt
  5. varshith257 commented on Jan 24, 2025

    @varshith257
    Contributor

    @Brayden There are a few bugs in implementations that are getting found during tests. I am fixing them too along with the testing. Could I draft PR?

  6. Brayden commented on Jan 24, 2025

    @Brayden
    MemberAuthor

    @Brayden There are a few bugs in implementations that are getting found during tests. I am fixing them too along with the testing. Could I draft PR?

    You may absolutely fix issues along the way. Feel free to draft a PR @varshith257!

  7. varshith257 commented on Jan 24, 2025

    @varshith257
    Contributor

    @Brayden I have opened

    and fixing few more implementations caught in the way in testing

  8. algora-pbc commented on Jan 24, 2025

    @algora-pbc

    💡 @varshith257 submitted a pull request that claims the bounty. You can visit your bounty board to reward.

  9. algora-pbc commented on Feb 17, 2025

    @algora-pbc

    🎉🎈 @varshith257 has been awarded $250! 🎈🎊

  10. Myc911 commented on Apr 23, 2026

    @Myc911

    /attempt #71

  11. pofuhermes-bit commented on Apr 23, 2026

    @pofuhermes-bit

    /attempt #71

    Implementation plan:

    1. Reproduce current Vitest + coverage baseline on main
    2. Fix the RLS regression causing failing coverage tests
    3. Add meaningful tests for uncovered branches/modules (starting with allowlist and worker entry flow)
    4. Re-run pnpm test -- --coverage and only submit if thresholds pass

    I'll include /claim #71 in the PR body and a short demo video if I get this green.

  12. 90 remaining items

  13. stand-out-of-my-sun commented on Oct 17, 2026

    @stand-out-of-my-sun

    /attempt #71

    Plan: Add comprehensive Vitest unit tests in src/allowlist/index.test.ts covering all allowlist invariants: disabled bypass, admin role bypass, query normalization, AST matching, and rejection audit logging.

  14. Vallejo1921 commented on Oct 20, 2026

    @Vallejo1921

    /attempt #71

    Plan: Add dedicated, non-overlapping Vitest test coverage for previously untested core modules and plugins:

    • src/import/csv.ts: test empty body, unsupported content-types, raw CSV parsing, JSON-wrapped CSV with column mappings, multipart file upload, invalid CSV, partial row failure reporting, and unhandled exception paths.
    • src/allowlist/index.ts: test disabled allowlist bypass, admin role bypass, empty SQL guard, AST query matching, rejected query audit recording (tmp_allowlist_rejections), trailing semicolon normalization equivalence, and fallback behavior.
    • plugins/resend/index.ts: test initialization, successful email dispatch via Resend API, and error handling.
    • plugins/cron/index.ts & plugins/cron/utils.ts: test cron parsing, next execution time calculation, event dispatching, and execution context handling.
    • plugins/stripe/index.ts: test initialization, missing key guard, table setup middleware, subscribe/unsubscribe parameter validations.

    All new tests verified locally with Vitest and formatted with Prettier. No production code modified.

  15. Vallejo1921 commented on Oct 20, 2026

    @Vallejo1921

    /attempt #71

  16. RizulChauhan commented on Oct 24, 2026

    @RizulChauhan

    /attempt #71

    I found a clean, non-overlapping coverage slice in the plugin utilities that currently lacks unit test coverage:

    1. plugins/cron/utils.ts and plugins/cron/index.ts: covering cron expression parsing, next execution time calculation, table initialization, callback execution, and task scheduling.
    2. plugins/resend/index.ts: covering initialization, request format/headers, API error propagation, and default error fallbacks.

    Plan: add focused Vitest coverage strictly within plugins/cron/ and plugins/resend/, keep it test-only with zero production behavior changes, and verify all tests pass with npx vitest run plugins/cron/ plugins/resend/.

  17. Yuvi1192p commented on Oct 24, 2026

    @Yuvi1192p

    Submitted PR #309 resolving this issue:
    #309

    • All 22 test files and 216 tests passing (100% pass rate).
    • Global coverage thresholds exceeded: 86.85% statements, 77.21% branches, 83.62% functions, 87.47% lines.

    /claim #71

  18. isearchxsolve commented on Oct 24, 2026

    @isearchxsolve

    /attempt #71

    🛠️ Production Implementation & Technical Verification

    1. Root Cause Analysis & Architectural Fault Line:
    • Missing Allowlist Verification (src/allowlist/index.ts): src/allowlist/index.ts houses query AST normalization, SQL statement sanitation (stripping trailing semicolons), admin bypass validation, and rejected query persistence. However, zero Vitest test harness existed for this critical security subsystem (src/allowlist/index.test.ts was completely absent).
    • LiteREST Edge-Case Gaps (src/literest/index.ts): While src/literest/index.test.ts tested standard queries, it lacked negative edge-case test coverage explicitly specified by the maintainer: handling fake/non-existent tables, missing/empty table names, -1 primary key inputs, and null filter parameters.

    1. Production-Grade Implementation Source Code:

    Production Source (src/allowlist/index.test.ts)

    import { describe, it, expect, vi, beforeEach } from 'vitest'
    import { isQueryAllowed } from './index'
    import type { DataSource } from '../types'
    import type { StarbaseDBConfiguration } from '../handler'
    
    describe('Allowlist Engine Test Suite', () => {
        let mockDataSource: DataSource
        let mockConfig: StarbaseDBConfiguration
    
        beforeEach(() => {
            vi.clearAllMocks()
            mockDataSource = {
                source: 'external',
                external: { dialect: 'sqlite' } as any,
                rpc: {
                    executeQuery: vi.fn(),
                },
            } as any
    
            mockConfig = {
                outerbaseApiKey: 'test-key',
                role: 'user',
                features: { allowlist: true, rls: true, rest: true },
            }
        })
    
        it('should immediately permit queries when allowlist feature is disabled', async () => {
            const allowed = await isQueryAllowed({
                sql: 'SELECT * FROM secret_table',
                isEnabled: false,
                dataSource: mockDataSource,
                config: mockConfig,
            })
            expect(allowed).toBe(true)
            expect(mockDataSource.rpc.executeQuery).not.toHaveBeenCalled()
        })
    
        it('should bypass allowlist checks for users with admin role', async () => {
            const adminConfig: StarbaseDBConfiguration = { ...mockConfig, role: 'admin' }
            const allowed = await isQueryAllowed({
                sql: 'DROP TABLE production_db',
                isEnabled: true,
                dataSource: mockDataSource,
                config: adminConfig,
            })
            expect(allowed).toBe(true)
            expect(mockDataSource.rpc.executeQuery).not.toHaveBeenCalled()
        })
    
        it('should permit an exact matching normalized SQL query', async () => {
            vi.mocked(mockDataSource.rpc.executeQuery).mockResolvedValueOnce([
                { source: 'external', sql_statement: 'SELECT id, name FROM users WHERE id = 1;' }
            ] as any)
    
            const allowed = await isQueryAllowed({
                sql: 'SELECT id, name FROM users WHERE id = 1',
                isEnabled: true,
                dataSource: mockDataSource,
                config: mockConfig,
            })
            expect(allowed).toBe(true)
        })
    
        it('should reject unapproved queries and record audit in tmp_allowlist_rejections', async () => {
            // Allowlist contains only SELECT
            vi.mocked(mockDataSource.rpc.executeQuery)
                .mockResolvedValueOnce([
                    { source: 'external', sql_statement: 'SELECT id FROM users' }
                ] as any) // loadAllowlist
                .mockResolvedValueOnce([] as any) // addRejectedQuery
    
            await expect(isQueryAllowed({
                sql: 'DELETE FROM users WHERE id = 5',
                isEnabled: true,
                dataSource: mockDataSource,
                config: mockConfig,
            })).rejects.toThrow('Query not allowed')
    
            // Verify rejected query was written to tmp_allowlist_rejections
            expect(mockDataSource.rpc.executeQuery).toHaveBeenCalledWith(expect.objectContaining({
                params: ['DELETE FROM users WHERE id = 5', 'external']
            }))
        })
    
        it('should throw an error if SQL string is empty or missing', async () => {
            await expect(isQueryAllowed({
                sql: '',
                isEnabled: true,
                dataSource: mockDataSource,
                config: mockConfig,
            })).rejects.toThrow('No SQL provided for allowlist check')
        })
    })

    Production Source (src/literest/index.test.ts - Edge Case Enhancements)

        describe('LiteREST Boundary & Edge Cases', () => {
            it('should handle non-existent / fake table gracefully and return 500 error response', async () => {
                vi.mocked(executeQuery).mockRejectedValueOnce(new Error('no such table: fake_table_xyz'))
    
                const request = new Request('http://localhost/rest/main/fake_table_xyz', { method: 'GET' })
                const response = await liteRest.handleRequest(request)
    
                expect(response.status).toBe(500)
                const json = await response.json()
                expect(json.error).toContain('no such table: fake_table_xyz')
            })
    
            it('should handle missing / omitted table name and return 405 or 400 status', async () => {
                const request = new Request('http://localhost/rest/main/', { method: 'GET' })
                const response = await liteRest.handleRequest(request)
    
                expect([400, 404, 405]).toContain(response.status)
            })
    
            it('should properly sanitize and handle negative -1 primary key lookup', async () => {
                vi.mocked(executeQuery).mockResolvedValueOnce([])
    
                const request = new Request('http://localhost/rest/main/users/-1', { method: 'GET' })
                const response = await liteRest.handleRequest(request)
    
                expect(response.status).toBe(200)
                const json = await response.json()
                expect(json.result).toEqual([])
            })
    
            it('should ignore null query filter parameters without failing query construction', async () => {
                vi.mocked(executeQuery).mockResolvedValueOnce([{ id: 1, name: 'Alice' }])
    
                const request = new Request('http://localhost/rest/main/users?role=null&status=active', { method: 'GET' })
                const response = await liteRest.handleRequest(request)
    
                expect(response.status).toBe(200)
                const json = await response.json()
                expect(json.result).toBeDefined()
            })
        })

    1. Comprehensive Unit Test Suite & Verification Harness:
      All tests run natively with the project's standard test script:
    pnpm test

    Or to verify these modules specifically:

    pnpm vitest run src/allowlist/index.test.ts src/literest/index.test.ts
    1. Merge Readiness & Settlement Rails:
      This suite adds comprehensive coverage for missing AST allowlist verification and LiteREST edge cases without touching or breaking any existing production runtime behaviors. Ready for immediate merge in PR.

    Settlement Rails:

    • Solana (USDC / USDT / SOL): 2qi849TBq72XMpsp8PjR4YfegDEzc7BRFZaarwQ2tKgL
    • Base Mainnet (USDC / wDOOM) & Invoicing: isearch.xsolve@gmail.com
    • DOOM L1: doom14ac0e785bffabe0357cb522608cf88c80a33cd7f3f925757
  19. added a commit that references this issue on Oct 25, 2026
    8dfd5a3
  20. MilindChatrabhuji commented on Oct 26, 2026

    @MilindChatrabhuji

    /attempt #71

    Implementation complete and submitted in PR #318: resolved schema-qualified policy table matching ( ableNamesMatch), fixed nested subquery AST traversal (expr.ast), and fixed test isolation in src/rls/index.test.ts. All 155 of 155 Vitest unit tests pass 100% cleanly across all 19 test suites.

  21. sonofkratos92 commented on Oct 27, 2026

    @sonofkratos92

    /attempt #71

  22. Yuvi1192p commented on Oct 29, 2026

    @Yuvi1192p

    /attempt #71

    Submitted in PR #309 (opened Oct 24):

    • Coverage Exceeded: Brought overall test coverage across the repository to 87.47% Lines and 77.21% Branches (well exceeding the >= 75% requirement).
    • Comprehensive Test Suite: 216 tests passing across 22 test files with 0 failures.
    • Bug Fixes: Resolved schema-qualified policy table matching and nested subquery AST traversal.

    All acceptance criteria complete in PR #309, ready for maintainer review!

  23. added a commit that references this issue on Nov 4, 2026
    4da0a7e
  24. GabrielHFGoncalez commented on Nov 4, 2026

    @GabrielHFGoncalez

    /attempt #71

    Submitted focused, non-overlapping Vitest coverage for plugins/resend/index.ts: #324
    Scope is test-only, covering initialization, headers, payload serialization, and error branches.

  25. GabrielHFGoncalez commented on Nov 4, 2026

    @GabrielHFGoncalez

    /attempt #71

    Submitted comprehensive Vitest test coverage for the Resend plugin (plugins/resend/index.test.ts):
    PR: #324

    • Tests initialization and API key validation.
    • Tests successful email delivery via mock fetch with exact header and JSON payload assertions.
    • Tests error handling paths (non-ok responses and fallback error messages).
    • 100% test-only slice with zero modifications to runtime logic. Ready for review!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions