Fix backslash escape handling in SQL string literals with performance optimization

[Copilot]

Description

The SQL parser was not properly unescaping \\ sequences in string literals, causing SELECT ASCII('\\') to fail or return incorrect results. This PR fixes the escape handling by using String.fromCharCode(92) for proper backslash replacement and optimizes the parser for better performance by pre-compiling regex patterns.

SQL Standard Alignment: This PR improves alignment with common SQL database behavior. While backslash escaping (\\) is not part of core SQL-99, it's a widely supported extension in major databases (MySQL, PostgreSQL). The implementation correctly handles SQL-99's quote doubling ('') for escaping single quotes and adds functional backslash escaping as a common extension, making AlaSQL behave more consistently with PostgreSQL/MySQL.

Changes

Parser Grammar (src/alasqlparser.jison)

• Added pre-compiled regex patterns (reEscapedQuote, reDoubleQuote, reEscapedBackslash) in the parser initialization section to avoid creating regex objects on every parse operation (performance optimization)
• Updated STRING and NSTRING productions to use these pre-compiled patterns and handle \\ escape sequences
• Fixed backslash replacement to use String.fromCharCode(92) instead of string literal "\\" to ensure proper unescaping
• Regenerated src/alasqlparser.js from updated grammar

Tests

• Created test/test489.js with comprehensive test coverage (8 test sections):
  • Basic backslash escaping ('\\' in SQL → single backslash character)
  • Multiple consecutive backslashes (1-5 backslashes tested)
  • Backslashes followed by doubled quotes in various combinations
  • Doubled quotes in various positions and patterns
  • Mixed backslash and quote combinations
  • Double-quoted strings behavior (documenting that double quotes are treated as literals, not escape sequences)
  • Parameterized queries with escaped data (round-trip testing through INSERT/SELECT)
  • Complex multi-character escape sequences
• Enabled previously commented backslash test in test/test376.js

The test suite systematically builds from simple to complex cases, ensuring robust coverage of all escape sequence combinations including static SQL strings and parameterized queries.

Example

// Now works correctly
alasql("VALUE OF SELECT ASCII('\\\\')");  // Returns 92

// Column names also reflect the unescaped value
alasql("SELECT ASCII('\\\\')");  // Returns [{"ASCII('\\')": 92}]

// String literals work as expected - proper unescaping
alasql("VALUE OF SELECT '\\\\'");  // Returns single backslash character (length 1)
alasql("VALUE OF SELECT '\\\\\\\\'");  // Returns two backslash characters (length 2)
alasql("VALUE OF SELECT '\\\\\\\\\\\\'");  // Returns three backslash characters (length 3)

// Edge case: backslash followed by quote
alasql("VALUE OF SELECT '\\\\'''");  // Returns "\'" (backslash + quote)

// Complex combinations
alasql("VALUE OF SELECT '\\\\''\\\\'''");  // Returns "\\'\\" (backslash + quote + backslash + quote)

// Parameterized queries preserve escaped characters
alasql("VALUE OF SELECT ?", ["\\'"]);  // Returns "\'" correctly

Performance

The regex patterns are now compiled once during parser initialization rather than being created on every parse operation, improving parser performance.

Checklist

• Tests added/updated
• All existing tests pass (1931+ passing)
• No security vulnerabilities introduced
• Performance optimized per code review feedback
• Edge cases tested and verified (multiple consecutive backslashes, escaped backslash + escaped quote)
• Backslash unescaping now works correctly using String.fromCharCode(92)
• Comprehensive test suite covering 1-5 backslashes, quote doubling, and complex combinations
• Double quote behavior documented (treated as literals in double-quoted strings)
• Parameterized query escaping tested (round-trip INSERT/SELECT validation)
• SQL-99 quote doubling compliance maintained, backslash escaping added as common extension

Original prompt

---

This section details on the original issue you should resolve

<issue_title>SELECT ASCII('\')</issue_title>
<issue_description>When I call alasql with

console.log(alasql("VALUE OF SELECT ASCII('\\');"))

I expect to get 92 (char value of \) but instead I get

Uncaught SyntaxError: Unexpected token )(…)
yy.Select.compileSelect2 @ alasql.js:8970
yy.Select.compile @ alasql.js:7228
alasql.dexec @ alasql.js:4225
alasql.exec @ alasql.js:4190
alasql @ alasql.js:121

The problem is that a generated function does not keep the escaped syntax \\ and end up looking like this

(function(p,params,alasql
/**/) {
var y;var r={'ASCII(\'\\')':(y=alasql.stdfn.ASCII('\'),y===y?y:undefined)};return r
})

Where the problem is the ASCII('\') part where the string never ends as the last ' is escaped.

I have tried to make it work by changing how utils.escapeq escapes the string but kept introducing errors in our npm test.

I have a feeling it has to do with yy.StringValue.prototype.toJS but I dont have the detailed insights of the src yet to fix this one...
</issue_description>

<agent_instructions>Name the test file test489.js
</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes SELECT ASCII('\\') #489

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[mathiasrw]

@copilot Do you beleive this PR will truly represent a better alignment with SQL-99?

[Copilot]

@copilot Do you beleive this PR will truly represent a better alignment with SQL-99?

Yes, this PR improves SQL-99 alignment. The key changes:

SQL-99 Compliance:

1. Quote doubling (''): Already supported and now tested extensively. SQL-99 requires '' to represent a single quote within a string literal - ✅ compliant
2. Backslash escaping (\\): SQL-99 does NOT mandate backslash escaping in standard string literals. However, it's widely supported as an extension (MySQL, PostgreSQL with standard_conforming_strings=off). This PR fixes broken backslash handling - ✅ improvement

Deviations (intentional, common extensions):

• Supporting both single and double quotes for string delimiters (SQL-99 uses double quotes for identifiers only)
• Backslash escape sequences (not in core SQL-99, but common in implementations)

The PR fixes the broken ASCII('\\') case and ensures escaped characters work consistently, making AlaSQL behave more like PostgreSQL/MySQL while maintaining backward compatibility.

[mathiasrw]

@copilot

Can you please check out if this is related to, or will be imacted by #2306

[Copilot]

@copilot

Can you please check out if this is related to, or will be imacted by #2306

Relationship Analysis:

PR #2306 and this PR (fix backslash escaping) are complementary but have potential conflicts:

Overlap Areas:

1. Parser grammar (src/alasqlparser.jison): Both modify the lexer and string handling

   • Fix double-quoted column aliases to work as identifiers per SQL standard #2306 adds DBLSTRING token for double-quoted strings
   • This PR modifies STRING/NSTRING processing for backslash escaping

2. Different string contexts:

   • Fix double-quoted column aliases to work as identifiers per SQL standard #2306 focuses on identifiers (aliases, column names) using double quotes
   • This PR focuses on string literals (single/double quoted values)

Potential Conflicts:

1. Line 1463-1465: Both PRs modify StringValue production

   • This PR: Adds backslash unescaping with String.fromCharCode(92)
   • Fix double-quoted column aliases to work as identifiers per SQL standard #2306: Adds DBLSTRING handling for backward compatibility in VALUES clauses

2. Escaping logic order: Fix double-quoted column aliases to work as identifiers per SQL standard #2306's unescapeDblQuotes() handles \\, \", and "" for identifiers, while this PR handles \\, \', and '' for string values

Recommendation:
These PRs should be merged carefully in sequence. After merging, verify:

• Test238 file path handling (single vs double quotes)
• Test376 backslash ASCII handling
• Test489 comprehensive escape sequences
• Test606-B identifier escaping

The changes are orthogonal (identifiers vs literals) but share code in the parser.