SDK for writing lookout analyzers
Project description
lookout SDK
What is lookout SDK?
Lookout SDK is a toolkit for writing new analyzers for the lookout.
What does it provide
It provides to an analyzer an easy access to the DataService API though a gRPC service.
DataService abstracts all data access and details of dealing with actual Git repositories, UAST extraction, programming language detection, etc. This way the analyzer can focus focus only on source code analysis logic.
The architecture of lookout and its components are described in src-d/lookout/docs
SDK includes:
- proto definitions
- pre-generated libraries code for Golang and Python
- low-level helpers to workaround protobuf/gRPC caveats. Check go documentation
- quickstart documentation on how to write an analyzer
How to create a new Analyzer
Essentially, every analyzer is just a gRPC server that implements Analyzer service. Lookout itself acts as a gRPC client for this server and it will push the analyzer server whenever a new Pull Request is ready for analysis.
Golang
Steps:
- use pre-generated code for gRPC from
gopkg.in/src-d/lookout-sdk.v0/pb
- implement Analyzer service interface. Example:
NotifyReviewEvent(ctx context.Context, review *pb.ReviewEvent) (*pb.EventResponse, error) NotifyPushEvent(context.Context, *pb.PushEvent) (*pb.EventResponse, error)
- analyzer should request a stream of files and UASTs from DataService that lookout exposes, by default, on
localhost:10301
- analyzer has options to ask either for all files, or just the changed ones, as well as UASTs, language, full file content and/or exclude some paths: by regexp, or just all vendored paths
- analyzer has to return a list of Comment messages
- analyzer should request a stream of files and UASTs from DataService that lookout exposes, by default, on
- run gRPC server to listen for requests from the lookout
SDK contains a quickstart example of an Analyzer that detects language and number of functions for every file language-analyzer.go:
go get -u .
go run language-analyzer.go
Python
pip install lookout-sdk
- use pre-generated code for gRPC from
lookout_sdk
library - implement Analyzer class that extends AnalyzerServicer. Example:
def NotifyReviewEvent(self, request, context): def NotifyPushEvent(self, request, context):
- analyzer should request a stream of files and UASTs from DataService that lookout exposes, by default, on
localhost:10301
- analyzer has options to ask either for all files, or just the changed ones, as well as UASTs, language, full file content and/or exclude some paths: by regexp, or just all vendored paths
- analyzer has to return a list of Comment messages
- analyzer should request a stream of files and UASTs from DataService that lookout exposes, by default, on
- start grpc server and add Analyzer instance to it
SDK contains a quickstart example of an Analyzer that detects language and number of functions for every file language-analyzer.py:
python3 language-analyzer.py
How to test analyzer
One can test analyzer locally without the need for Github access and a full lookout server installation using lookout-sdk
binary. You can think about it as curl-like tool to call an analyzer gRPC endponts. For convenience, it also exposes a DataServer backed by a git repository in local FS.
- get
lookout-sdk
binary from src-d/lookout releases - run
bblfshd
- build and start analyzer e.g. Golang
go get -u .
go run language-analyzer.go
, or Pythonpip install lookout-sdk
python3 language-analyzer.py
- test without Github access, on the latest commit in some Git repository in local FS
$ lookout-sdk review \ --log-level=debug \ --git-dir="$GOPATH/src/gopkg.in/src-d/lookout-sdk.v0" \ "ipv4://localhost:2020"
this will create a "mock" Review event and notify the analyzer, as if you were creating a Pull Request from HEAD~1
.
Check src-d/lookout for further details on lookout-sdk
binary CLI options.
Extracting UAST from arbitrary code
Lookout server and lookout-sdk
binary proxy bblfsh gRPC protocol through DataServer.
Please check how to create client in bblfsh documentation and use DataServer address to connect.
Caveats
- client: disable secure connection on dialing with
grpc.WithInsecure()
- client: set max gRPC message size:
- go: use
pb.DialContext
instead. - python: use
lookout.sdk.grpc.create_channel
instead.
- go: use
- client: turn off gRPC fail-fast mode If your analyzer greedy creates a connection to DataServer before one was actually started, you might want to disable fail-fast mode. This way the RPCs are queued until the chanel ready. Here is an example.
- client/server RFC 3986 URI scheme support:
- go: use
pb.ToGoGrpcAddress
andpb.Listen
- python: use
lookout.sdk.grpc.to_grpc_address
- go: use
Release Process
- Make sure the code is up to date using
make protogen
. - Update
VERSION
inpython/setup.py
with the same version that you will use for the tag (manual step required until #2 is implemented). - Create the release tag.
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for lookout_sdk-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d09b860757100831a27558aa513bf675a82085e71bc7ab0fe4677c1f6a09f77b |
|
MD5 | c2ab0d726841de13e815aa1475a21cc6 |
|
BLAKE2b-256 | 4b9b82ac281538abc1929611db540c6bbfbd8952901508625d1a20a9fc120109 |