metpx-sr3 3.0.15

Subscribe, Acquire, and Re-Advertise products.

MetPX-Sarracenia

page web / homepage: https://metpx.github.io/sarracenia

[ Guides ] [ Manual Pages ]	[ Orientation ] [ Référence ]
MetPX-sr3 (Sarracenia v3) is a data duplication or distribution pump that leverages existing standard technologies (web servers and the AMQP brokers) to achieve real-time message delivery and end-to-end transparency in file transfers. Data sources establish a directory structure which is carried through any number of intervening pumps until they arrive at a client. The client can provide explicit acknowledgement that propagates back through the network to the source. Whereas traditional file switching is a point-to-point affair where knowledge is only between each segment, in Sarracenia, information flows from end-to- end in both directions. At its heart, Sarracenia exposes a tree of web accessible folders (WAF), using any standard HTTP server (tested with apache). Weather applications are soft real-time, where data should be delivered as quickly as possible to the next hop, and minutes, perhaps seconds, count. The standard web push technologies, ATOM, RSS, etc… are actually polling technologies that when used in low latency applications consume a great deal of bandwidth and overhead. For exactly these reasons, those standards stipulate a minimum polling interval of five minutes. Advanced Message Queueing Protocol (AMQP) messaging brings true push to notifications, and makes real-time sending far more efficient.	MetPX-sr3 (Sarracenia v3) est un engin de copie et de distribution de données qui utilise des technologies standards (tel que les services web et le courtier de messages AMQP) afin d’effectuer des transferts de données en temps réel tout en permettant une transparence de bout en bout. Alors que chaque commutateur Sundew est unique en soit, offrant des configurations sur mesure et permutations de données multiples, Sarracenia cherche à maintenir l’intégrité de la structure des données, tel que proposée et organisée par la source jusqu’à destination. Le client peut fournir des accusés de réception qui se propagent à travers tous les noeuds de la chaîne, en sens inverse jusqu’à la source. Tandis qu’un commutateur traditionnel échange les données de point à point, Sarracenia permet le passage des données d’un bout à l’autre du réseau, tant dans une direction que dans l’autre. Sarracenia, à sa plus simple expression, expose une arborescence de dossiers disponibles sur la toile (“Web Accessible Folders”). Le temps de latence est une composante névralgique des applications météo: les minutes, et parfois les secondes, sont comptées. Les technologies standards, telles que ATOM et RSS, sont des technologies qui consomment beaucoup de bande passante et de ressouces lorsqu’elles doivent répondre à ces contraintes. Ces standards limitent la fréquence maximale de vérification du serveur à cinq minutes. Le protocole de séquencement de messages avancés (Advanced Message Queuing Protocol, AMQP) est une approche beaucoup plus efficace pour la livraison d’annonces de nouveaux produits.
An initiative of Shared Services Canada http://ssc-spc.gc.ca in support of internal needs of the Government of Canada	Une initiative de Services partagés Canada https://ssc-spc.gc.ca pour appuyer les opérations du gouvernement du Canada

Concepts: Sarracenia in 10 Minutes Video (anglais seulement)

Release Notes

lists all changes between versions.

git repo

v03_wip … still in development, too many changes to list. unstable, breaking changes possible, only for experimental use.

3.00.15b2

• #490 implemented CI/CD matrix doing flow tests with multiple python versions.

• #489 sourceFromExchange was missing implementation.

• #488 there was a problem with recovering from connection failures.

• #487 fixed to ignore unjustified flow test failures.

• #486 problem with inflight.

• #483, #455, #479 debian packaging working again, based on use of extras (optional components.)

• improved messaging of mdelaylatest flow callback.

• large, numerous improvements in the documentation (approaching release quality.)

• tls_rigour becomes tlsRigour (in implementation, was only documented that way before.)

• #480 sr3 status display problem not ignoring files that are not .conf ones.

• #477 sr3 edit of default.conf, credentials.conf, admin.conf work again.

3.00.14.b1

• forked off from v2.

• has MQTT support.

• just incrementing minor release as dev.

• non-stable releases for now.

• now in beta: no more breaking changes expected.

• 3.x is a deep re-factor vs. 2.x

• for more information, see new web-site: https://metpx.github.io/sarracenia

Peter Silva <[e-mail]>

Project Prime mover, Manager & Evangelist. A lot of Documentation, and Review of Docs. Much discussion with Michel. Small bug fixes. wrote none of the code in v2, except most (all?) plugins included with package. added the ‘plugin’ model to v2. Main blame for sr3.

Cléa Aumont for refactoring of FTP processing to fix timezone support.

porting many plugins from v2 to sr3. Resolved issued with xattr python modules several fixes to status… hmm… how about issues: #447, #438, #435, #433, #433, #381, #378, #366, #350

Reid Sunderland Added SASL authentication and vhost support.

Extensive work getting flow tests working with sr3. worked on some refactoring as well.

Cody Au fixes #413, #387, #348. working on sr process management.

Michel Grenier <[e-mail]> (Retired)

dd_subscribe, sr_subscribe, sr_sarra, sr_post, All of the code until 2016/03. A lot of the code, until 2017/08-2018/05 (major refactoring.) Retired now… none of the code since sr3, but a lot was copy/pasted.

Jun Hu <[e-mail]>

Documentation Diagrams, lead on some deployments (head tester!) plugin work.

Noureddine Habili <[e-mail]>

lead on other deployments ( project lead on Sarrasemina ) plugin work.

Benoit Lapointe <[e-mail]>

Improving testing frameworks/ Design documentation / code hygiene. Some MQTT interop work.

Khosrow Ebrahimpour <[e-mail]> (no long involved.)

Packaging & Process (Debian, Launchpad, some pypi, the vagrant self-test) originally proposed migration to github.

Daluma Sen <[e-mail]>

some work on sr_watch, and worked on sr_post as well for caching.

Murray Rennie <[e-mail]>

sr_winnow, worked on that with Michel. lots of work on plugins and LDM bridge for UNIDATA and NOAAPORT ingest.

Dominic.Racette <[e-mail]>

actually a client, not really an author, but participation was crucial. for five years of patient testing in a very large deployment scenario.

credit for Sarracenia image used as github avatar photo

Jay Lechtman [e-mail] - www.horrorculture.weebly.com