cyares 0.4.0

A Cython Version of c-ares

An Upgraded version of pycares with faster and safer features.

Installation

How to install

pip install cyares

How to install with Trio bundle

pip install cyares[trio]

How to install with anyio bundle

pip install cyares[anyio]

How to install with optional IDNA Plugin

pip install cyares[idna]

How to install with optional aiohttp extension

pip install cyares[aiohttp]

Installing all bundles

pip install cyares[all]

Small Examples of How to Use

from cyares import Channel

# NOTE: Dns Queries contain a Future object
# This is a little bit different from pycares 
# Handles were used to prevent new or unwanted vulnerabilities along
# with easier control given to the user...
# The Future object comes from the python concurrent.futures standard library

def main():
    with Channel(servers=["8.8.8.8", "8.8.4.4"], event_thread=True) as channel:
        data = channel.query("google.com", "A").result()
    print(data)

if __name__ == "__main__":
    main()

In Asyncio

There's a module for an aiodns version if aiodns were using cyares.

import winloop # or uvloop (asyncio will also work) 
from cyares.aio import DNSResolver

async def test():
    async with DNSResolver(["8.8.8.8", "8.8.4.4"]) as dns:
        data = await dns.query("google.com", "A")
    print(data)

if __name__ == "__main__":
    winloop.run(test())

Trio

Trio is now supported as well which means this library is not just limited to synchronous python or asyncio.

from cyares.trio import DNSResolver
import trio

async def test():
    async with DNSResolver(["8.8.8.8", "8.8.4.4"], rotate=True, event_thread=False) as dns:
        result = await dns.query("google.com", "A")
        print(result)

if __name__ == "__main__":
    trio.run(test)

Anyio

As of 0.4.0 anyio is now supported if you need use an anyio implementation

from cyares.anyio import DNSResolver
import anyio

async def test():
    async with DNSResolver(["8.8.8.8", "8.8.4.4"], rotate=True, event_thread=False) as dns:
        result = await dns.query("google.com", "A")
        print(result)

if __name__ == "__main__":
    anyio.run(test)

Aiohttp

Using Globally (Monkey Patch Method)

[!WARNING] aiohttp implementation is still buggy. Use at your own risk.

install() applies a monkeypatch globally Know that if you wanted to use different resolvers with aiohttp you can do that by passing CyAresResolver through to the ClientSession. another way to avoid install() not being threadsafe is by installing before the rest of your code begins running. Same as how uvloop and winloop both work

from cyares.aiohttp import install
from aiohttp import ClientSession
import asyncio

async def main():
    # CyAres should be monkeypatched to be the default 
    # DNS Resolver for this http request
    async with ClientSession() as client:
        async with client.get("https://httpbin.org/ip") as result:
            data = await result.json()
            print(f'YOUR IP ADDRESS IS: {data["origin"]}')

if __name__ == "__main__":
    install()
    asyncio.run(main())

when your done with an evenloop run and want to rerun another but with an alternate dnsresolver such as aiodns or the ThreadedResolver use uninstall()

Using Non-Globally

Although slightly more tricky it is possible to pass along CyAresResolver with the right setup without resorting to install(). This will also work with aiohttp_socks or using aiohttp_socks with a proxy or tor or i2p. If you feel uncomfortable with sending this much stuff to ClientSession yourself it's encouraged to use globally with install() or via monkeypatching aiohttp's default dns resolver in your own way.

from cyares.aiohttp import CyAresResolver
from aiohttp import ClientSession, TCPConnector

# NOTE: This should also be respected when aiohttp_socks is in use
# Such as the ProxyConnector which is another connector type of it's
# own

async def main():
    async with ClientSession(
        connector=TCPConnector(CyAresResolver()
    )) as client:
        async with client.get("https://httpbin.org/ip") as result:
            data = await result.json()
            print(f'YOUR IP ADDRESS IS: {data["origin"]}')

In Cython

from cyares cimport Channel, Future, AresError


def dns_resolve_a(str domain):
    """Resolves IPV4 Using cython"""
    cdef Future fut
    with Channel(['8.8.8.8', '8.8.4.4'], event_thread=True) as   channel:
        fut = cannel.query(domain, "A")
    return fut.wait()

Story

As Someone who as recently started to contribute to projects such as aiohttp and a few of it's smaller libraries. The asynchronous dns resolver aiodns that aiohttp can optionally use, felt like the oddest one in the group. With aiodns using pycares under the hood I wanted to learn how it worked to see if I could use my skills to optimize it like I had previously done with propcache and multidict as well but I soon came to learn of it's many problems and when pycares started to hang on me with lingering threads when it ran, something didn't feel right to me and it lead me down a very large rabbit-hole waiting to be re-explored.

Pycares was quick, dirty and fast but as the years of it's existance went by something wasn't exactly right with it and it could use safer features, better optimization and safety practices. While cffi might be both quick and easy to use I didn't see the benefit of using it. The many vulnerabilities it was getting told me something needed to change if not big. With Pycares reaching version 5.0.0 I wanted to celebrate it the same way node-js celebrated the 10 year annversery of the original http-parser. Write a new one.

The Rewrite

While contributing to the aio-libs repos whenever I had a gap of free-time on hand while waiting on a question to be answered and when my other python libraries felt stable enough such as (deprecated-params, aiocallback, aiothreading & winloop) this is where I spent my time on. It had a total of 3 re-writes until I was satisfied with the result. Turns out just doing everything in small chunks makes all the difference.

It only felt right to me that migrating the library from cffi to cython would be the best solution to the problem. Not Rust or C, just pure-cython and small amounts of C whenever nessesary and re-changing many of the internals to incorperate a safer approch. The reason for not picking Rust is that there wasn't nessesarly a need to be memory-safe. What I cared about was speed, when people are doing DNS Lookups, speed matters and Rust did not have a friendly enough archetecture that would set a future or allow me to access the parent from the child.

There was no better canadate than to move to using handles the same way uvloop & winloop do it and having authored one of them I was familliar with this concept.

The idea was not to reinvent the wheel rather move parts of the library over in chunks and look at pycares for clues on how certain things should be laid out. At the end of the day, Pycares was the blueprint and using concurrent.futures & Cython was the cure. Py_buffer techniques brought over from msgspec were utilized incase users were planning to use any bytes or string objects of any sort and I made sure the license was on it incase users wanted to know where it came from.

If there was a deprecated function with an alternative & safer approch to use I felt using it would be a better move. For example, servers are now set with the csv functions rather than the original setup and because of that change you could now set dns servers in url formats and I might possibly look at adding in yarl to be the cherry on top for those who might wish to get real creative about dns server urls.

Moving over the ares_query function to use the dns recursion function is planned for the future (maybe after I do the first release of cyares). I still wanted to retain the original response data it gives so that migrating from pycares to cyares wouldn't be a pain to anyone who planned on moving and I also didn't want to take away from pycares either if you prefer using cffi over cython the same way curl-cffi and cycurl were both done. I have my fingers crossed that aiodns will adopt this library in the future or allow users to choose between pycares and cyares.