share-df 1.5.2

Share and Edit Pandas/Polars Dataframes with a Link!

share-df: Instantly Share and Modify Dataframes With a Web Interface From Anywhere

              Demos and Source Code

Problem

Data scientists use Pandas/Polars dataframes to work with their data, but nontechnical contributors need a GUI like Excel. This is frustrating because:

1. The developers have to manage the back and forth for converting to Excel, downloading the file, uploading it back into thier IDE, and then reformatting the data.
2. The code previously written to work with the DF may fail if during the conversion process the formatting changes signficantly
3. It becomes difficult to manage a large amount of edits and different versions of the Excel files especially if the developers are using dataframe versioning tools like Weights and Biases Artifact logging.

Goal

Developers generate a URL for free that they can then send to nontechnical contributors enabling them to modify the dataframe with a web app.

Technical Contributor Features

• one function call to generate a link to send, accessible anywhere
• changes made by the client are received back as a dataframe for seamless development
• compatale for both pandas and polars dataframes
• local option for seeing/presenting dataframe visually in browser

Nontechnical Contributor Features

• Easy Google OAuth login
• Seamless UI to modify the dataframe
  • Rename columns (Shift Click)
  • Drag around columns
  • Modify values
  • Add new columns and rows
• Send the results back with the click of a button
• Work with large amounts of data quickly
• Multiple collaborator support
  • See which cells other collaborators are editing
  • Sync all changes other collaborators live
• See version history, undo changes, restore versions.

How to Run

1. pip install share-df
2. If you do not already have one, generate an auth token for free in less than a minute with ngrok
3. Create a .env file in your directory with NGROK_AUTHTOKEN=
4. import and call the function on any df!

Example Code

import pandas as pd
from share_df import pandaBear

df = pd.DataFrame({
    'Name': ['John', 'Alice', 'Bob', 'Carol'],
    'City': ['New York', 'London', 'Paris', 'Tokyo'],
    'Salary': [50000, 60000, 75000, 65000]
})

df = pandaBear(df)
print(df)

Handling Big Data

• As per the demo, currently, the site takes 6 seconds to load a million rows.
• After loading, it can handle cell changes, row additions, column sorting, new columns, fast scrolling, and sending the data back frictionlessly.
• That being said given interest I can optimize this.

Google Colab

• This code works by creating a localhost and then tunneling traffic to make it accessible to other people.
• Thereby, since Google Colab code runs on a VM this is an interesting challenge to handle.
• As of 0.1.7 the package offers support for creating a Google-generated link for DFs but this link is not shareable.
• For Google Colab instead of using a .env I recommend putting your NGROK_AUTHTOKEN into the Google Colab secrets manager (key icon on the left side of the screen). That way your secrets also can be synced to other notebooks and you don't have to repeat the .env uploading each time.
• I initially aimed for full functionality (link sharing) with Google Colab however it seems impossible as Colab locks it to Colab session authentification.
• Google has also stated that they may deprecate their serve_kernel_port_as_window function in the future in which case it will be swapped to serve_kernel_port_as_iframe and the same functionality will remain except it will be in the IFrame.
• For now, there is an optional parameter that allows you to use the editor in IFrame mode.
• Check out a demo notebook here.

Community Requested Features (eg. from the reddit thread)

☑ 3rd option for discarding changes (completed as of 1.1.0)

☑ FastAPI template for easier maintenance

☑ Multiple authenticated users