Squerly takes the tedium out of nested dicts and lists.
Project description
Overview
Make digging around in nests of python dicts and lists less tedious.
Motivation
Say you have a pile of yaml docs like this (some data snipped for brevity):
Scheduler:
apiVersion: operator.openshift.io/v1 kind: KubeScheduler metadata: annotations: release.openshift.io/create-only: 'true' creationTimestamp: 2019-09-09 06:16:52+00:00 generation: 1 name: cluster resourceVersion: '20599' selfLink: /apis/operator.openshift.io/v1/kubeschedulers/cluster uid: 6a531a9d-d2c9-11e9-b8a1-005056be2a3b spec: managementState: Managed status: conditions: - lastTransitionTime: 2019-09-09 06:41:57+00:00 status: 'False' type: MonitoringResourceControllerDegraded latestAvailableRevision: 6 latestAvailableRevisionReason: '' nodeStatuses: - currentRevision: 6 nodeName: control-plane-0 - currentRevision: 6 nodeName: control-plane-2 - currentRevision: 5 nodeName: control-plane-1 targetRevision: 6 readyReplicas: 0
ControllerManager
apiVersion: operator.openshift.io/v1 kind: KubeControllerManager metadata: annotations: release.openshift.io/create-only: 'true' creationTimestamp: 2019-09-09 06:16:52+00:00 generation: 3 name: cluster resourceVersion: '17773' selfLink: /apis/operator.openshift.io/v1/kubecontrollermanagers/cluster uid: 6a4badba-d2c9-11e9-b8a1-005056be2a3b spec: logLevel: '' managementState: Managed observedConfig: extendedArguments: cloud-config: - /etc/kubernetes/static-pod-resources/configmaps/cloud-config/config cloud-provider: - some_cloud cluster-cidr: - 10.128.0.0/14 cluster-name: - some-test-fgt78 feature-gates: - ExperimentalCriticalPodAnnotation=true - RotateKubeletServerCertificate=true - SupportPodPidsLimit=true - LocalStorageCapacityIsolation=false service-cluster-ip-range: - 172.30.0.0/16 serviceServingCert: certFile: /etc/kubernetes/static-pod-resources/configmaps/service-ca/ca-bundle.crt operatorLogLevel: '' status: - lastTransitionTime: 2019-09-09 06:41:57+00:00 status: 'False' type: MonitoringResourceControllerDegraded latestAvailableRevision: 6 latestAvailableRevisionReason: '' nodeStatuses: - currentRevision: 6 nodeName: control-plane-2 - currentRevision: 6 nodeName: control-plane-1 - currentRevision: 6 nodeName: control-plane-0 readyReplicas: 0
APIServer
apiVersion: operator.openshift.io/v1 kind: KubeAPIServer metadata: annotations: release.openshift.io/create-only: 'true' creationTimestamp: 2019-09-09 06:16:52+00:00 generation: 4 name: cluster resourceVersion: '20635' selfLink: /apis/operator.openshift.io/v1/kubeapiservers/cluster uid: 6a5ecbb4-d2c9-11e9-b8a1-005056be2a3b spec: logLevel: '' managementState: Managed observedConfig: admission: pluginConfig: network.openshift.io/ExternalIPRanger: configuration: allowIngressIP: false apiVersion: network.openshift.io/v1 kind: ExternalIPRangerAdmissionConfig network.openshift.io/RestrictedEndpointsAdmission: configuration: apiVersion: network.openshift.io/v1 kind: RestrictedEndpointsAdmissionConfig restrictedCIDRs: - 10.128.0.0/14 - 172.30.0.0/16 apiServerArguments: cloud-config: - /etc/kubernetes/static-pod-resources/configmaps/cloud-config/config cloud-provider: - some_cloud feature-gates: - ExperimentalCriticalPodAnnotation=true - RotateKubeletServerCertificate=true - SupportPodPidsLimit=true - LocalStorageCapacityIsolation=false authConfig: oauthMetadataFile: /etc/kubernetes/static-pod-resources/configmaps/oauth-metadata/oauthMetadata corsAllowedOrigins: - //127\.0\.0\.1(:|$) - //localhost(:|$) imagePolicyConfig: internalRegistryHostname: image-registry.openshift-image-registry.svc:5000 servicesSubnet: 172.30.0.0/16 servingInfo: namedCertificates: - certFile: /etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt keyFile: /etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key - certFile: /etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt keyFile: /etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key - certFile: /etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt keyFile: /etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key - certFile: /etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt keyFile: /etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key storageConfig: urls: - https://etcd-0.example.com:2379 - https://etcd-1.example.com:2379 - https://etcd-2.example.com:2379 operatorLogLevel: '' unsupportedConfigOverrides: null status: conditions: - lastTransitionTime: 2019-09-09 06:34:21+00:00 message: 3 nodes are active; 1 nodes are at revision 2; 2 nodes are at revision 5 status: 'True' type: Available - lastTransitionTime: 2019-09-09 06:21:53+00:00 message: 1 nodes are at revision 2; 2 nodes are at revision 5 status: 'True' type: Progressing latestAvailableRevision: 5 latestAvailableRevisionReason: '' nodeStatuses: - currentRevision: 2 nodeName: control-plane-1 targetRevision: 5 - currentRevision: 5 nodeName: control-plane-0 - currentRevision: 5 nodeName: control-plane-2 readyReplicas: 0
And you want to know which ones mention node statuses not at the latest revision.
You could do it like this:
def get_stale(resources): results = [] for doc in resources: for status in doc.get("status", {}).get("nodeStatuses", []): if "targetRevision" in status and status["targetRevision"] != status["currentRevision"]: results.append(doc) return results scheduler = load("scheduler.yaml") cm = load("controller_manager.yaml") api = load("api_server.yaml") stale = get_stale([scheduler, cm, api])
Or you could do it like this:
from squerly import List, convert, Queryable scheduler = convert(load("scheduler.yaml")) cm = convert(load("controller_manager.yaml")) api = convert(load("api_server.yaml")) docs = Queryable(List([scheduler, cm, api])) stale = docs.status.nodeStatuses.where(lambda s: s.currentRevision != s.targetRevision).roots
Or better yet, run ./analyze.py must-gather.local.12345/
and get access to
every resource in the archive:
In [1]: stale = conf.status.nodeStatuses.where(lambda s: s.currentRevision != s.targetRevision).roots In [2]: stale.kind Out[2]: - kind: KubeScheduler - kind: KubeAPIServer In [3]: conf.find(("message", matches("Perm"))) Out[3]: - message: 'rm: cannot remove ''/etc/cni/net.d/80-openshift-network.conf'': Permission denied ' - message: 'rm: cannot remove ''/etc/cni/net.d/80-openshift-network.conf'': Permission denied ' - message: '+ source /run/etcd/environment /bin/sh: line 3: /run/etcd/environment: Permission denied ' In [8]: conf.find(("message", matches("Perm"))).upto("items").metadata.name Out[8]: - name: sdn-7llq6 - name: etcd-member-control-plane-0
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
squerly-0.2.2.linux-x86_64.tar.gz
(18.1 kB
view hashes)
Built Distribution
squerly-0.2.2-py3-none-any.whl
(14.9 kB
view hashes)
Close
Hashes for squerly-0.2.2.linux-x86_64.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 440691ad1842a01253b56cdcbebd96f916abc18251071cfb4ad0650aacda5dbc |
|
MD5 | eab8f3793d323e00b6ec5d1a98d67d0c |
|
BLAKE2-256 | 6379b5f728e8cf79f513cdca26aa7098b3b23c0ae5b4caccd5482a8024b3c87a |