1. Find Your Python Installation:
   To locate your Python 3.12 executable, run:

   py -3.12 -c "import sys; print(sys.executable)"
   

2. Create a New Virtual Environment:
   Using the path found above, create a new empty venv:

   C:/Users/path/to/Programs/Python/Python312/python.exe -m venv test_env
   

3. Activate the Virtual Environment:

   source test_env/Scripts/activate
   

4. Verify the Active Environment:
   Since there is no direct equivalent to conda env list, you can check your active environment via:

   echo $VIRTUAL_ENV
   

1. Create a New Conda Environment:
   Replace 3.12 with your desired Python version if necessary.

   conda create -n new_env python=3.12
   

2. Activate the Environment:

   conda activate new_env
   

gr_libs also includes a library of trained agents for the various supported environments within the package.

To download the dataset of trained agents and caches, use the new CLI tool after installing gr_libs:

download-gr-libs-dataset

You can also specify a custom extraction directory:

download-gr-libs-dataset --extract_to /path/to/target/dir

This will download and extract the required files into the specified directory (by default, the package directory).

You may also use the following alternative methods:

• Clone the repository and run the script directly:

  git clone https://github.com/MatanShamir1/gr_libs.git
  cd gr_libs
  pip install gdown
  python download_dataset.py
  

• Manual download from Google Drive:

  • Download gr_cache.zip
  • Download trained_agents.zip

  After downloading, unzip both files into your desired directory within the package.

  After extraction, you should observe the folders trained_agents and gr_cache in your current working directory. These folders contain the necessary datasets for running experiments and using the provided tools.

• Use our docker image, which includes the dataset in it: First, make sure you have docker downloaded, installed and that the docker daemon is running. Then,

1. pull the image:

docker pull ghcr.io/matanshamir1/gr_image:latest

2. run a container:

docker run -it ghcr.io/matanshamir1/gr_image:latest bash

3. You end up with a container that has the datasets in it, but doesn't have the packages. Install the package from within the container, go back to 'Setup' for that.

1. Create a recognizer

   Specify the domain name and specific environment for the recognizer, effectively telling it the domain theory - the collection of states and actions in the environment.

   import gr_libs.environment # Triggers gym env registration - you must run it!
   recognizer = Graql(
       domain_name="minigrid",
       env_name="MiniGrid-SimpleCrossingS13N4"
   )
   

2. Domain Learning Phase (For GRAQL)

   GRAQL does not accumulate information about the domain or engage in learning activities during this phase. Other algorithms don't require any data for the phase and simply use what's provided in their intialization: the domain and environment specifics, excluding the possible goals.

3. Goal Adaptation Phase

   The recognizer receives new goals and corresponding training configurations. GRAQL trains goal-directed agents and stores their policies for inference.

   recognizer.goals_adaptation_phase(
       dynamic_goals=[(11,1), (11,11), (1,11)],
       dynamic_train_configs=[(QLEARNING, 100000) for _ in range(3)]  # For expert sequence generation
   )
   

4. Inference Phase

   This phase generates a partial sequence from a trained agent, simulating suboptimal behavior with Gaussian noise.

   actor = TabularQLearner(
       domain_name="minigrid",
       problem_name="MiniGrid-SimpleCrossingS13N4-DynamicGoal-11x1-v0",
       algorithm=QLEARNING,
       num_timesteps=100000
   )
   actor.learn()
   full_sequence = actor.generate_observation(
       action_selection_method=stochastic_amplified_selection,
       random_optimalism=True  # Adds noise to action values
   )
   partial_sequence = random_subset_with_order(full_sequence, int(0.5 * len(full_sequence)), is_consecutive=False)
   closest_goal = recognizer.inference_phase(partial_sequence, (11,1), 0.5)
   

5. Evaluate the result

   print(f"Closest goal returned by Graql: {closest_goal}\nActual goal actor aimed towards: (11, 1)")
   

The consts.py file contains predefined ODGR problem configurations. You can use existing configurations or define new ones.

To execute an ODGR problem using the configuration file, you specify a recognizer, a domain, a gym environment within that domain and the task:

python odgr_executor.py --recognizer ExpertBasedGraml --domain minigrid --task L1 --env_name MiniGrid-SimpleCrossingS13N4

If you also add the flag:

 --collect_stats

to the cmd, 3 kinds of outputs will be generated from the ODGR problem's execution: a. Into:

outputs\\minigrid\MiniGrid-SimpleCrossingS13N4\MiniGrid-SimpleCrossingS13N4\L1\experiment_results

a .pkl and a .txt summary in a dictionary format will be generated, including the summary of all ODGR executions, including runtime and overall accuracies for all lengths and types of input sequences.

b. Into:

outputs\ExpertBasedGraml\minigrid\MiniGrid-SimpleCrossingS13N4\policy_sequences\MiniGrid-SimpleCrossingS13N4-DynamicGoal-1x11-v0_inference_seq/plan_image.png

a visulzation of the sequence the agent generated will be dumped, either in a png or an mp4 format, depending on the domain, for debugability.

c. Into: either:

outputs\ExpertBasedGraml\minigrid\MiniGrid-SimpleCrossingS13N4\goal_embeddings

In Graml algorithms, or:

outputs\Graql\minigrid\MiniGrid-SimpleCrossingS13N4\confidence

In GRAsRL algorithms, pickled results from which confidence of the results can be obtained, for offline analysis.

For GRAsRL outputs, for every possible goal, the likelihood of it being the true goal from the input sequence, based on the policy distance metric.

For GRAML outputs, the embeddings of the sequences are pickled for every goal-directed sequence. Offline, since, since in the embdding space of GRAML's metric model- sequences towards the same sequences are close and vice versa, one could reproduce the most likely goal by measuring the elementwise vector distance of the embeddings, and retrieve a confidence of it.