airflow grpc operator
Project description
Airflow Grpc Operator
Fork on Airflow 2.0
Fix Airflow 1.10.x use grpc operator
Requirements:
apache-airflow
grpcio
protobuf
How to install:
pip install airflow-grpc
How to use:
- airflow 1.10.x
from airflow_grpc.grpc_operator import GrpcOperator
args = {
'owner': 'Airflow',
'start_date': days_ago(1),
}
dag = DAG(
dag_id='dag_id',
default_args=args,
schedule_interval=None
)
def callback(response: Any, **context):
return response
run_this = GrpcOperator(task_id='task_id',
dag=dag,
grpc_conn_id='your_grpc_connection_id_on_admin_connections',
stub_class=YOUR_GRPC_STUB_CLASS,
call_func='your_grpc_stub_function',
request_data_func=YOUR_GRPC_MESSAGE_FOR_REQUEST,
response_callback=YOUR_RESPOSNE_METHOD,
xcom_task_id='XCOM_TASK_ID',
data=YOUR_REQUEST_DATA_DICT
)
- airflow 2.0.0
First must implement the stub_class method
import xxx_pb2_grpc, xxx_pb2
from utils.grpc_hook import BaseGrpcHook
class xxxGrpcHook(BaseGrpcHook):
def stub_class(self, channel):
return xxx_pb2_grpc.xxxServiceStub(channel)
Use the implemented class
xxxGrpcHook().run('MethodName', {'request': xxx_pb2.xxxRequestMethod(**data_dict)})
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
airflow-grpc-0.0.10.tar.gz
(3.9 kB
view hashes)
Built Distribution
Close
Hashes for airflow_grpc-0.0.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea6655a3baf359fd10e4ad1a661ec9fa124714f1c85caca404cd7087476ecae3 |
|
MD5 | 91e2dbcdc3b9bbd83a176366deca26bc |
|
BLAKE2b-256 | 8a3b246a14d84db37797336eb5ca40ac693bf69d1e51621b0c9b3b35b9acd15b |