Modular SQL Linting for Humans
SqlFluff :scroll: :black_nib: :sparkles:
The SQL Linter for humans
Bored of not having a good SQL linter that works with whichever dialiect you're working with? Fluff is an extensible and modular linter designed to help you write good SQL and catch errors and bad SQL before it hits your database.
Sqlfluff is still in an open alpha phase - expect the tool to change significantly over the coming months, and expect potentially non-backward compatable api changes to happen at any point. In particular moving from 0.0.x to 0.1.x introduced some non backward compatible changes and potential loss in functionality. If you'd like to help please consider contributing.
To get started just install the package, make a sql file and then run sqlfluff and point it at the file.
$ pip install sqlfluff $ echo " SELECT a + b FROM tbl; " > test.sql $ sqlfluff lint test.sql == [test.sql] FAIL L: 1 | P: 1 | L003 | Single indentation uses a number of spaces not a multiple of 4 L: 1 | P: 14 | L006 | Operators should be surrounded by a single space unless at the start/end of a line L: 1 | P: 27 | L001 | Unnecessary trailing whitespace
For more details on usage see the docs on readthedocs here.
There's lots to do in this project, and we're just getting started. If you want to understand more about the architecture of sqlfluff, you can find more here.
All notable changes to this project will be documented in this file.
[0.2.4] - 2019-12-06
--code-onlyoption to the
parsecommand to spit out a more simplified output with only the code elements.
- Rules can now optionally override the description of the violation
and pass that back via the
- Bugfix, correct missing files in
- Better parsing of the not equal operator.
- Added more exclusions to identifier reserved words to fix cross joins.
- At verbosity levels 2 or above, the root config is printed and then any diffs to that for specific files are also printed.
- Linting and parsing of directories now reports files in alphabetical order. Thanks @barrywhart.
- Better python 2.7 stability. Thanks @barrywhart.
- Fixing parsing of
[0.2.3] - 2019-12-02
- Bugfix, default config not included.
[0.2.2] - 2019-12-02
- Tweek rule L005 to report more sensibly with newlines.
- Rework testing of rules to be more modular.
- Fix a config file bug if no root config file was present for some values. Thanks @barrywhart.
- Lexing rules are now part of the dialect rather than a global so that they can be overriden by other dialects when we get to that stage.
[0.2.0] - 2019-12-01
- Templating support (jinja2, python or raw).
- Variables + Macros.
fixcommand is also sensitive to fixing over templates and will skip certain fixes if it feels that it's conflicted.
- Config file support, including specifying context for the templater.
- Documentation via Sphinx and readthedocs.
- Including a guide on the role of SQL in the real world. Assisted by @barrywhart.
- Documentation LINTING (given we're a linting project) introduced in CI.
- Reimplemented L006 & L007 which lint whitespace around operators.
- Ability to configure rule behaviour direclty from the config file.
- Implemented L010 to lint capitalisation of keywords.
- Allow casting in the parser using the
ORDER BYusing indexes and expressions.
- Added parsing of
- Support for window/aggregate functions.
- Added linting and parsing of alias expressions.
- Fixed a bug which could cause potential infinite recursion in configuration
- Changed how negative literals are handled, so that they're now a compound segment rather than being identified at the lexing stage. This is to allow the parser to resolve the potential ambiguity.
- Restructure of rule definitions to be more streamlined and also enable
autodocumentation. This includes a more complete
RuleSetclass which now holds the filtering code.
- Corrected logging in fix mode not to duplicate the reporting of errors.
- Now allows insert statements with a nested
- Fixed verbose logging during parsing.
- Allow the
Bracketedgrammar to optionally match empty brackets using the optional keyword.
[0.1.5] - 2019-11-11
- Python 3.8 Support!
- Moved some of the responsibility for formatted logging into the linter to mean that we can log progressively in large directories.
- Fixed a bug in the grammar where one of the return values was messed up.
[0.1.4] - 2019-11-10
- Added a
--exclude-rulesargument to most of the commands to allow rule users to exclude specific subset of rules, by @sumitkumar1209
- Added lexing for
- Added a new common segment:
LambdaSegmentwhich allows matching based on arbitrary functions which can be applied to segments.
- Recursive Expressions for both arithmetic and functions, based heavily off the grammar provided by the guys at CockroachDB.
Anythinggrammar, useful in matching rather than in parsing to match anything.
- Complete rewrite of the bracket counting functions, using some centralised class methods
BaseGrammarclass to support common matching features across multiple grammars. In particular this affects the
Delimitedgrammar which is now much simpler but does also require slightly more liberal use of terminators to match effectively.
- Rather than passing around multiple variables during parsing and matching, there is now
ParseContextobject which contains things like the dialect and various depths. This simplifies the parsing and matching code significantly.
- Bracket referencing is now done from the dialect directly, rather than in individual
Grammars (except the
Bracketedgrammar, which still implements it directly). This takes out some originally duplicated code.
- Corrected the parsing of ordering keywords in and
- Removed the
bracket_sensitive_forward_matchmethod from the
BaseGrammar. It was ugly and not flexible enough. It's been replaced by a suite of methods as described above.
[0.1.3] - 2019-10-30
- Tweak to the L001 rule so that it doesn't crash the whole thing.
[0.1.2] - 2019-10-30
- Fixed the errors raised by the lexer.
[0.1.1] - 2019-10-30
- Fixed which modules from sqlfluff are installed in the setup.py. This affects
[0.1.0] - 2019-10-29
- Big Rewrite - some loss in functionality might be apparent compared to pre-0.1.0. Please submit any major problems as issues on github
- Changed unicode handling for better escape codes in python 2. Thanks @mrshu
- BIG rewrite of the parser, completely new architecture. This introduces
breaking changes and some loss of functionality while we catch up.
- In particular, matches now return partial matches to speed up parsing.
Delimitedmatcher has had a significant re-write with a major speedup and broken the dependency on
- Rewrite of
Sequenceto use partial matches properly.
- Different treatment of numeric literals.
Delimitedrespect bracket counting.
- MASSIVE rewrite of
- Grammars now have timers.
- Joins properly parsing,
- Rewrite of logging to selectively output commands at different levels
of verbosity. This uses the
- Added a command line
sqlfluff parseoption which runs just the parsing step of the process to better understand how a file is being parsed. This also has options to configure how deep we recurse.
- Complete Re-write of the rules section, implementing new
crawlerswhich implement the linting rules. Now with inbuilt fixers in them.
- Old rules removed and re implemented so we now have parity with the old rule sets.
- Moved to using Ref mostly within the core grammar so that we can have recursion.
- Used recursion to do a first implementation of arithmetic parsing. Including a test for it.
- Moved the main grammar into a seperate dialect and renamed source and test files accordingly.
- Moved to file-based tests for the ansi dialect to make it easier to test using the tool directly.
- As part of file tests - expected outcomes are now encoded in yaml to make it easier to write new tests.
- Vastly improved readability and debugging potential of the _match logging.
- Added support for windows line endings in the lexer.
[0.0.7] - 2018-11-19
- Added a
sqlfluff fixas a command to implement auto-fixing of linting errors. For now only
L001is implemented as a rule that can fix things.
- Added a
rulescommand to introspect the available rules.
- Updated the cli table function to use the
testwraplibrary and also deal a lot better with longer values.
- Added a
--rulesargument to most of the commands to allow rule users to focus their search on a specific subset of rules.
- Refactor the cli tests to use the click CliRunner. Much faster
[0.0.6] - 2018-11-15
- Number matching
- Fixed operator parsing and linting (including allowing the exception of
[0.0.5] - 2018-11-15
- Much better documentation including the DOCS.md
- Fixed comma parsing and linting
[0.0.4] - 2018-11-14
- Added operator regexes
- Added a priority for matchers to resolve some ambiguity
- Added tests for operator regexes
- Added ability to initialise the memory in rules
[0.0.3] - 2018-11-14
- Refactor of rules to allow rules with memory
- Adding comma linting rules (correcting the single character matchers)
- Adding mixed indentation linting rules
- Integration with CircleCI, CodeCov and lots of badges
- Changed import of version information to fix bug with importing config.ini
- Added basic violations/file reporting for some verbosities
- Refactor of rules to simplify definition
- Refactor of color cli output to make it more reusable
[0.0.2] - 2018-11-09
- Longer project description
- Proper exit codes
- colorama for colored output
- Significant CLI changes
- Much improved output from CLI
[0.0.1] - 2018-11-07
- Initial Commit! - VERY ALPHA
- Restructure into package layout
- Adding Tox and Pytest so that they work
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.