srcsrv 1.2.15

Source indexing package

Source Indexing

Table of Content

• Source Indexing
  • Table of Content
  • Disclaimer
  • Package Description
  • Usage
    • Common options
    • Indexing operation
    • Source fetching operation
    • Common diagnostic options
    • Build diagnostic options
  • Setup and environment
• Plugins
  • Github options
  • Github environment
  • Bitbucket options
  • Bitbucket environment
  • Gitlab options
  • Gitlab environment
  • Codebase options
  • Codebase environment
  • Options file
• External Links

Disclaimer

The author of this package does not warrant that the functionality contained in the package will meet your requirements or that the operation of the package will be uninterrupted or error-free. Note: In no event will the author be liable to you for any damages, including any corruption of binaries or PDBs, lost profit, lost savings, lost patience or other incidental or consequential damage.

With that part out of the way, my goal is to make something that is useful. If you'd like to request additional features, report bugs or provide any other feedback, feel free to reach me.
Uri Mann

Package Description

This package provides two distinct operations needed for source indexing functionality.

1. Python script to add source indexing information to .PDB files. The added information allows the debugger to automatically pull the version of the source code used at build time from a repository. The python script can be invoked on each .PDB file after the link phase of the build is completed. Alternatively, the script can receive a list of one or more directories where the .PDBs are placed at the end of the build. Internally, the script simply scans each directory recursively and invoke itself on each file with .pdb extension.
2. Python script to fetch the source code files from the remote repository and cache them for use by the debugger. That script is invoked by the debugger when it detects that the source code for next line is not cached on the debugging host machine.

Usage

Note that some of the command line options are common to both index and fetch operation while others are operation specific. Additionally, each plugin has specific additional operational options as well. Some options have default values which will be used if the option is omitted.
The option and the value may be separated by a space or equal sign (=) (e.g., both --log output.log and --log=output.log are the same).

Common options

-$, --plugin - Plugin class. default is srcsrv.plugins.Github. Plugin classes available in current release: srcsrv.plugins.Github, srcsrv.plugins.Bitbucket, srcsrv.plugins.Codebase or srcsrv.plugins.Gitlab.
-@, --uri - Git repository server URI. default github.com
-#, --commit - Build commit hash or label. This option can be omitted. If not provided the package is assuming the current directory is the repository and the uses current HEAD hash.
-!, --action - Plugin action with choice between index or fetch (default fetch).

Indexing operation

-p, --pdbs - List of paths to .PDB files to process or directories to scan for symbol files (e.g.: -p c:\path\file1.pdb c:\path\file2.pdb d:\symdir1 d:\symdir2). The script will recurs to each sub-directory in the specified list. The path is assumed to be fully-qualified or relative to current directory.
-b, --build-base - Root of the build directory. This path correspond with top of the repository branch being built (i.e., where .git is placed).
-x, --extensions - Semicolon separated list of source extensions (default:cpp;hpp;c;h).
-s, --srcsrv - Path to SDK or DDK source indexing directory. Default path is C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\srcsrv (Windows 10 DTfW or newer required). Download and install Debugging Tools for Windows.

Source fetching operation

-c, --cache - Source files cache directory. If no value passed as an argument, it defaults to user's home directory (%USERPROFILE%). All source files retrieved from the repository will be stored in a subdirectory name .srcsrv.
-v, --verify - Should the script do SSL certificate validation of the remote repository. Possible options are True, False, 0, 1 or None (default: None).
-e, --python - Path of python executable used to for running the script. The default is py.exe (Python launcher).

Common diagnostic options

-l, --log - Path to log file. By default logging is not enabled. The dash (-) will send logging to stdout.
-d, --debug - Allows attaching remote debugger to the script execution. This option is using debugpy local connection on port 5678.

Build diagnostic options

-k, --keep - Keep (don't delete after processing) the .PDB stream input file. With this option specified the file is kept in the same directory as the .PDB. The file will have the same base name as the .PDB file with the extension .ini (e.g., prog.pdb will have corresponding prog.ini).
-n, --dry-run - The script is run without modifying the .PDB. While it is possible to index the same symbols file multiple times it is not recommended. Each operation adds another srcsrv stream instead of overwriting the existing one. This option should be used with with --keep option when setting for diagnosing source indexing for the first time.

Setup and environment

This package depends on GitPython and requests packages. Both should be automatically installed by pip when you install this package. Some additional set up is a perquisite to proper function of this package. This split between setting for index and fetch operation (see: --action argument).
Git SCM - The Git software must be installed for both indexing as well as debugging with this package.
Debugging Tools for Windows - Must be installed on the host where the indexing takes place. These tools are already included in any debugger installation so no additional steps are needed.
Python - This package requires Python version 3.8 or above.

GIT_PYTHON_GIT_EXECUTABLE - Add this variable to your environment if git.exe is not in the path.
GIT_PYTHON_TRACE - Add this variable to your environment to allow GitPython to add tracing to the log.
SRCSRV_INI_FILE - For debugging you can optionally add SRCSRV.ini file. See more on this topic in the advance section.
Plugins environment - Each of the different repos plugin may require additional environment variables to be set. See the section bellow for more information on specific repos.

Plugins

Each plugin has unique name of an environment variable containing the authentication used to connect to the remote repository. The value stored in the environment variable will be evaluated as python language expression. It is expected to be in one of three types. See more information in the plugins section.

1. Dictionary (dict) entry which is interpreted as HTTPS Authorization header (e.g., {'Authorization':'Bearer <token>'}).
2. Tuple (tuple) value which in interpreted as HTTPS basic authentication (e.g., ('<account>','<password>')).
3. The value is missing or None to indicate no authentication is required by the remote repository.

Github options

-u, --account - User's account (default: %SRCSRV_USERNAME%).
-r, --repo - Github repository name.

Github environment

SRCSRV_GITHUB_AUTH - User token This is expected to be the string in the format of {'Authorization':'Token <token>'}. The value is evaluated as a python language expression.

Bitbucket options

-i, --api - Version of Bitbucket REST API used. The default is 2.0.
-t, --project-key - Project key name.
-r, --repo-slug - Repository slug name.

Bitbucket environment

SRCSRV_BITBUCKET_AUTH - User token. This is expected to be the string in the format of {'Authorization':'Bearer <token>'}. The value is evaluated as a python language expression.

Gitlab options

-u, --account - User's account (default: %SRCSRV_USERNAME%).
-t, --project-id - Project id value.

-i, --api - Version of Gitlab REST API used. The default is v4.

Gitlab environment

SRCSRV_GITLAB_AUTH - Token used for authorization This is expected to be the string in the format of {'Authorization':'Bearer <token>'}. The value is evaluated as a python language expression.
. It may be one of three token types.

1. Personal access tokens.
2. Project access tokens
3. Group access tokens

Codebase options

-u, --account - User's account (default: %SRCSRV_USERNAME%).
-i, --domain - Codebase repository domain.
-t, --proj-permalink - Project perma link.
-r, --repo-permalink - Repository perma link.
-j, --api - REST API version (default is api3).

Codebase environment

SRCSRV_CODEBASE_AUTH - Basic authentication used for authorization. This is expected to be the string in the format of ('<domain>/<account>':'<token>'). The value is evaluated as a python language expression.

Options file

For indexing operation the script can also be invoked with a response file. Using @path\resp_file_name. The file can contain any of the above parameters. Response file and command line parameters can be combined. Example:

--action index
--build-base D:\dev\myproject
--uri bitbucket.com
--pdbs test.pdb
--api 1.0
--project-key myproj
--repo-slug testing
--commit release-1.0.0
--log ..\srcsrv.log
--plugin srcsrv.plugins.Bitbucket

Similar to the command line the option and argument may be separated by space, new line, or equal sign.

External Links

Note that these links may not work properly in the package site (https://pypi.org/project/srcsrv/). Go to https://github.com/urielmann/srcsrv-public for better results.
Srcsrv.doc
The SrcTool Utility
Source Indexing is Underused Awesomeness
Debugging with source indexed .PDB
Advance topics .PDB
Introduction to SRCSRV Package