Skip to main content

Binary Files Parser resulted from the script Logger from CRFS - Rfeye Node Equipament

Project description

RFPYE

Este módulo tem como objetivo o processamento e extração otimizada de dados dos arquivos .bin de monitoramento do espectro provenientes do script Logger executados nas estações de Monitoramento CRFS RFeye Node. Para tal utilizamos as várias funcionalidades da biblioteca fastcore, que expande e otimiza as estruturas de dados da linguagem python.

Instalação

Como parte dessa lib utiliza código c compilado com Cython, é preciso que um compilador C esteja instalado. No entanto é recomendado a criação de um ambiente virtual para que a instalação das dependências não interfira com o a instalação base do python. Para tal é recomendamos o uso do conda. A seguir é mostrado instruções para a criação do ambiente virtual, com todas as dependências, inclusive o compilador C, utilizando o conda.

Instale o miniconda. Com o conda instalado e disponível no seu PATH ou através do Anaconda Prompt, execute os comando:

Linux:

conda create -n rfpye pip python=3.7 gcc -c intel -c conda-forge -y
conda activate rfpye
python -m pip install rfpye

Windows

conda create -n rfpye pip python=3.7 libpython m2w64-toolchain -c intel -y
conda activate rfpye
echo [build] > %CONDA_PREFIX%\Lib\distutils\distutils.cfg
echo compiler = mingw32 >> %CONDA_PREFIX%\Lib\distutils\distutils.cfg
python -m pip install rfpye

O comando acima cria um ambiente virtual com o mesmo nome da biblioteca rfpye, instala as dependências básicas necessárias para a compilação, em seguida ativa o ambiente virtual e instala o módulo. As duas linhas adicionais em Windows cria um arquivo de configuração distutils.cfg para o compilador m2w64, que é utilizado pelo conda para compilar o código.

Depois disso basta instalar normalmente a lib: python -m pip install rfpye

Em Linux normalmente o sistema já possui o compilador gcc instalado então basta executar o comando pip install acima.

Como utilizar

Abaixo mostramos as funcionalidades principais dos módulos, utilizando-os dentro de algum outro script ou REPL

Precisamos necessariamente de um diretório de entrada, contendo um ou mais arquivos .bin

Mude os caminhos abaixo para suas pastas locais

from fastcore.xtras import Path
from rfpye.utils import get_files
from rich import print

A função abaixo baixa alguns arquivos de exemplo:

path = Path(r'binfiles')
if not path.exists() or not len(get_files(path, extensions=['.bin'])):
    path = Path('.')
    !wget --header 'Host: raw.githubusercontent.com' --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --header 'Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3' --referer 'https://github.com/EricMagalhaesDelgado/SpecFiles/blob/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002092_210208_T202310_CRFSBINv.5.bin' --header 'DNT: 1' --header 'Upgrade-Insecure-Requests: 1' 'https://raw.githubusercontent.com/EricMagalhaesDelgado/SpecFiles/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002092_210208_T202310_CRFSBINv.5.bin' --output-document 'rfeye002092_210208_T202310_CRFSBINv.5.bin'
    !wget --header 'Host: raw.githubusercontent.com' --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --header 'Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3' --referer 'https://github.com/EricMagalhaesDelgado/SpecFiles/blob/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002092_210208_T203131_CRFSBINv.2.bin' --header 'DNT: 1' --header 'Upgrade-Insecure-Requests: 1' 'https://raw.githubusercontent.com/EricMagalhaesDelgado/SpecFiles/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002092_210208_T203131_CRFSBINv.2.bin' --output-document 'rfeye002092_210208_T203131_CRFSBINv.2.bin'
    !wget --header 'Host: raw.githubusercontent.com' --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --header 'Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3' --referer 'https://github.com/EricMagalhaesDelgado/SpecFiles/blob/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002292_210208_T202215_CRFSBINv.4.bin' --header 'DNT: 1' --header 'Upgrade-Insecure-Requests: 1' 'https://raw.githubusercontent.com/EricMagalhaesDelgado/SpecFiles/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002292_210208_T202215_CRFSBINv.4.bin' --output-document 'rfeye002292_210208_T202215_CRFSBINv.4.bin'
    !wget --header 'Host: raw.githubusercontent.com' --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --header 'Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3' --referer 'https://github.com/EricMagalhaesDelgado/SpecFiles/blob/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002292_210208_T203238_CRFSBINv.3.bin' --header 'DNT: 1' --header 'Upgrade-Insecure-Requests: 1' 'https://raw.githubusercontent.com/EricMagalhaesDelgado/SpecFiles/main/Combo3%20(CRFS%20Bin%20-%20DataTypes%204%2C%207%2C%208%2C%2060-65%20e%2067-69)/rfeye002292_210208_T203238_CRFSBINv.3.bin' --output-document 'rfeye002292_210208_T203238_CRFSBINv.3.bin'

A função parse_bin é a função principal que encapsula o processamento dos arquivos bin.

parse_bin[source]

parse_bin(bin_file:Union[str, Path], precision=float32)

Receives a CRFS binfile and returns a dictionary with the file metadata, a GPS Class and a list with the different Spectrum Classes A block is a piece of the .bin file with a known start and end and that contains different types of information. It has several fields: file_type, header, data and footer. Each field has lengths and information defined in the documentation. Args: bin_file (Union[str, Path]): path to the bin file

Returns: Dictionary with the file metadata, file_version, string info, gps and spectrum blocks.

Extração de Dados

Vamos listar arquivos da última versão do script Logger, CRFS Bin - Versão 5

files = get_files(r'D:\OneDrive - ANATEL\Sensores', extensions=['.bin'])
file = files.shuffle()[0]
%%time
dados = parse_bin(file)
Wall time: 15.1 s
print(dados)
{
    'filename': 'rfeye002263_210618_T210255.bin',
    'file_version': 23,
    'string': 'CRFS DATA FILE V023',
    'hostname': 'rfeye002263',
    'method': 'ScriptRFeye2021_v.1',
    'unit_info': 'Stationary',
    'file_number': 0,
    'identifier': 'INFO',
    'gps': GPS Data - Median of Coordinates: -23.32905:-51.13701 Altitude: 591.30 
#Satellites: 11.0,
    'spectrum': [SpecData(type=67, thread_id=300, description='PMEC 2021 (Faixa 1 de 10).', 
start_mega=105.0, stop_mega=140.0, dtype='dBm', ndata=3584, bw=18457, processing='peak', 
antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=310, description='PMEC 
2021 (Faixa 2 de 10).', start_mega=155.0, stop_mega=165.0, dtype='dBm', ndata=1024, bw=18457,
processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=100,
description='PRD 2021 (Faixa principal 1 de 4).', start_mega=50.0, stop_mega=90.0, 
dtype='dBμV/m', ndata=1024, bw=73828, processing='peak', antuid=0, thresh=-45.5, 
minimum=-45.5), SpecData(type=67, thread_id=110, description='PRD 2021 (Faixa principal 2 de 
4).', start_mega=70.0, stop_mega=110.0, dtype='dBμV/m', ndata=1024, bw=73828, 
processing='peak', antuid=0, thresh=-29.5, minimum=-29.5), SpecData(type=67, thread_id=120, 
description='PRD 2021 (Faixa principal 3 de 4).', start_mega=170.0, stop_mega=220.0, 
dtype='dBμV/m', ndata=1280, bw=73828, processing='peak', antuid=0, thresh=-51.5, 
minimum=-51.5), SpecData(type=67, thread_id=130, description='PRD 2021 (Faixa principal 4 de 
4).', start_mega=470.0, stop_mega=700.0, dtype='dBμV/m', ndata=5888, bw=73828, 
processing='peak', antuid=0, thresh=-39.5, minimum=-39.5), SpecData(type=67, thread_id=200, 
description='PMEF 2021 (Faixa 1 de 6).', start_mega=700.0, stop_mega=960.0, dtype='dBm', 
ndata=6656, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), 
SpecData(type=67, thread_id=210, description='PMEF 2021 (Faixa 2 de 6).', start_mega=1710.0, 
stop_mega=1980.0, dtype='dBm', ndata=6912, bw=73828, processing='peak', antuid=0, 
thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=220, description='PMEF 2021 
(Faixa 3 de 6).', start_mega=2100.0, stop_mega=1002168.999, dtype='dBm', ndata=1792, 
bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, 
thread_id=230, description='PMEF 2021 (Faixa 4 de 6).', start_mega=2290.0, stop_mega=2390.0, 
dtype='dBm', ndata=2560, bw=73828, processing='peak', antuid=0, thresh=-147.5, 
minimum=-147.5), SpecData(type=67, thread_id=240, description='PMEF 2021 (Faixa 5 de 6).', 
start_mega=2500.0, stop_mega=2690.0, dtype='dBm', ndata=4864, bw=73828, processing='peak', 
antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=320, description='PMEC 
2021 (Faixa 3 de 10).', start_mega=320.0, stop_mega=340.0, dtype='dBm', ndata=512, bw=73828, 
processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=330,
description='PMEC 2021 (Faixa 4 de 10).', start_mega=400.0, stop_mega=410.0, dtype='dBm', 
ndata=256, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), 
SpecData(type=67, thread_id=340, description='PMEC 2021 (Faixa 5 de 10).', start_mega=960.0, 
stop_mega=1001218.999, dtype='dBm', ndata=6656, bw=73828, processing='peak', antuid=0, 
thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=350, description='PMEC 2021 
(Faixa 6 de 10).', start_mega=1001388.999, stop_mega=1001428.999, dtype='dBm', ndata=1280, 
bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, 
thread_id=360, description='PMEC 2021 (Faixa 7 de 10).', start_mega=1530.0, 
stop_mega=1001648.999, dtype='dBm', ndata=3072, bw=73828, processing='peak', antuid=0, 
thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=370, description='PMEC 2021 
(Faixa 8 de 10).', start_mega=2690.0, stop_mega=1002898.999, dtype='dBm', ndata=5376, 
bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, 
thread_id=380, description='PMEC 2021 (Faixa 9 de 10).', start_mega=5000.0, stop_mega=5160.0,
dtype='dBm', ndata=4096, bw=73828, processing='peak', antuid=0, thresh=-147.5, 
minimum=-147.5), SpecData(type=67, thread_id=390, description='PMEC 2021 (Faixa 10 de 10).', 
start_mega=1005338.999, stop_mega=1005458.999, dtype='dBm', ndata=3328, bw=73828, 
processing='peak', antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=67, thread_id=250,
description='PMEF 2021 (Faixa 6 de 6).', start_mega=3290.0, stop_mega=3700.0, dtype='dBm', 
ndata=10496, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5)]
}

A saída da função é um dicionário, com os metadados do arquivo.

GPS

No entanto as duas chaves mais importantes do dicionário retornado são gps e spectrum

Se você imprimir a classe retornada pela chave gps é retornado um resumo dos seus atributos:

print(dados['gps'])
GPS Data - Median of Coordinates: -23.32905:-51.13701 Altitude: 591.30 #Satellites: 11.0

Para extrair os atributos em si de dado objeto e retorná-los todos num dicionário, o módulo utils tem a função auxiliar getattrs

getattrs[source]

getattrs(obj:Any, attrs:Iterable[T_co]=None)

Receives an object and return the atributes listed in attrs, if attrs is None return its public attributes

print(getattrs(dados['gps']))
{'altitude': 807.8, 'latitude': -23.44242, 'longitude': -46.472522, 'num_satellites': 11.0}

Os atributos listados são os valores consolidados por meio da mediana dos diversos blocos de GPS do arquivo.

Dados Brutos de GPS

Caso desejar a lista original de valores, os atributos são os mesmos mas precedidos de _, o que os torna atributos privados em python, isso somente quer dizer que não são explicitados em algus métodos como getattrs, pois normalmente não são acessíveis diretamente, mas nada impede que sejam acessados.

dados['gps']._latitude
(#9058) [-23.329051,-23.329054,-23.329058,-23.329051,-23.329052,-23.329054,-23.32904,-23.329055,-23.329057,-23.32906...]
dados['gps']._longitude
(#9058) [-51.137013,-51.137008,-51.137002,-51.137009,-51.137008,-51.137005,-51.137015,-51.137019,-51.137025,-51.13702...]
dados['gps']._altitude
(#9058) [590.8,589.9,589.9,590.6,590.7,589.8,590.3,589.7,589.0,589.5...]
dados['gps']._num_satellites 
(#9058) [11,11,11,11,11,11,11,11,11,11...]

Dados de Nível Espectral

Cada arquivo bin normalmente possui vários fluxos de espectro distintos, cada fluxo espectral é uma classe Python, na chave spectrum é retornado uma lista com todos os fluxos de espectro.

fluxos = dados['spectrum']
print(len(fluxos))
20

Vamos investigar um deles:

fluxo = fluxos[0]

Ao imprimir um fluxo é mostrado informações mínimas sobre o seu conteúdo:

print(fluxo)
Blocks of Type: 67, Thread_id: 300, Start: 105.0 MHz, Stop: 140.0 MHz

A função repr retorna uma representação com todos os metadados do fluxo:

print(repr(fluxo))
SpecData(type=67, thread_id=300, description='PMEC 2021 (Faixa 1 de 10).', start_mega=105.0, 
stop_mega=140.0, dtype='dBm', ndata=3584, bw=18457, processing='peak', antuid=0, 
thresh=-147.5, minimum=-147.5)

Qualquer um dos atributos listados podem ser acessados diretamente:

print(fluxo.description) , print(fluxo.bw)
PMEC 2021 (Faixa 1 de 10).
18457
(None, None)

No entanto o principal atributo de um fluxo de espectro são os valores de nível medidos, os valores medidos são retornados por meio do atributo levels:

print(fluxo.levels)
[[ -95.   -87.5  -86.  ...  -91.5  -90.5  -91.5]
 [ -85.5  -86.   -87.  ...  -90.5  -92.   -96. ]
 [ -86.   -90.   -95.5 ... -100.   -91.   -91.5]
 ...
 [ -89.5  -90.   -85.5 ...  -96.5  -91.   -91. ]
 [ -91.   -94.   -92.5 ...  -91.   -96.5  -98.5]
 [ -83.   -83.5  -91.  ...  -99.   -99.5  -95.5]]
print(f'Formato da matriz com os níveis: {fluxo.levels.shape}')
Formato da matriz com os níveis: (2576, 1024)

O nº de linhas da matriz nos dá o número de pontos medidos naquele dado fluxo e as colunas o número de traços no qual o Span ( Stop - Start ) foi dividido. O número de traços pode ser retornada também por meio da função len

print(len(fluxo))
9058

O atributo anterior retorna uma numpy.ndarray, que é um formato eficiente para processamento.

Medidas de nível como pandas dataframe

No entanto temos adicionalmente o método .matrix() que retorna a matriz de dados como um Pandas Dataframe formatada com o tempo da medição de cada traço como índice das linhas e as frequências de cada traço como coluna.

Vamos mostrar as cinco primeiras e cinco últimas linhas e colunas.

fluxo.matrix().iloc[:5, :5]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 105.000000 105.009768 105.019537 105.029305 105.039073
Time
2021-06-18 21:03:00.762503 -95.0 -87.5 -86.0 -90.0 -92.5
2021-06-18 21:04:01.432664 -85.5 -86.0 -87.0 -84.0 -77.5
2021-06-18 21:05:01.182570 -86.0 -90.0 -95.5 -85.0 -84.5
2021-06-18 21:06:00.812513 -86.0 -87.5 -91.5 -96.5 -93.5
2021-06-18 21:07:01.402635 -90.0 -83.0 -84.5 -88.5 -85.5
fluxo.matrix().iloc[-5:, -5:]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 139.960927 139.970695 139.980463 139.990232 140.000000
Time
2021-07-02 18:56:00.731317 -96.0 -97.0 -104.0 -100.5 -102.5
2021-07-02 18:57:00.731239 -99.5 -99.5 -101.0 -101.0 -98.5
2021-07-02 18:58:00.731138 -95.0 -99.5 -99.5 -101.5 -98.5
2021-07-02 18:59:00.731242 -99.5 -99.5 -97.0 -97.5 -98.5
2021-07-02 19:00:00.732413 -95.5 -99.5 -98.0 -97.5 -103.5

Novamente, caso desejado acessar todos os atributos de um fluxo no formato de dicionário, basta utilizar a função getattrs

print(getattrs(fluxo))
{
    'antuid': 0,
    'bw': 18457,
    'description': 'PMEC 2021 (Faixa 1 de 10).',
    'dtype': 'dBm',
    'minimum': -147.5,
    'ndata': 3584,
    'processing': 'peak',
    'start_mega': 105.0,
    'stop_mega': 140.0,
    'thread_id': 300,
    'thresh': -147.5,
    'type': 67
}

CRFS Bin - Versão 5 - Arquivos Comprimidos

Vamos listar arquivos da última versão do script Logger, Versão 5, arquivos comprimidos onde o piso de ruído é suprimido.

file = r'binfiles\compressed\rfeye002290_210922_T204046_MaskBroken.bin'
%%time
compressed = parse_bin(file)
Wall time: 8.76 s
print(compressed)
{
    'filename': 'rfeye002290_210922_T204046_MaskBroken.bin',
    'file_version': 23,
    'string': 'CRFS DATA FILE V023',
    'hostname': 'rfeye002290',
    'method': 'ScriptRFeye2021_v2.cfg',
    'unit_info': 'Stationary',
    'file_number': 0,
    'identifier': 'INFO',
    'gps': GPS Data - Median of Coordinates: -14.81416:-39.03184 Altitude: 15.50 #Satellites:
11.0,
    'spectrum': [SpecData(type=68, thread_id=321, description='PMEC 2021 (Faixa 3 de 10). @ 
-80dBm, 100kHz.', start_mega=320.0, stop_mega=340.0, dtype='dBm', ndata=512, bw=73828, 
processing='peak', antuid=0, thresh=-100, minimum=-147.5), SpecData(type=68, thread_id=301, 
description='PMEC 2021 (Faixa 1 de 10). @ -80dBm, 10kHz.', start_mega=108.0, stop_mega=137.0,
dtype='dBm', ndata=14848, bw=3690, processing='peak', antuid=0, thresh=-100, minimum=-147.5),
SpecData(type=68, thread_id=341, description='PMEC 2021 (Faixa 5 de 10). @ -80dBm, 100kHz.', 
start_mega=960.0, stop_mega=1001218.999, dtype='dBm', ndata=6656, bw=73828, 
processing='peak', antuid=0, thresh=-100, minimum=-147.5), SpecData(type=68, thread_id=311, 
description='PMEC 2021 (Faixa 2 de 10). @ -80dBm, 10kHz.', start_mega=156.0, stop_mega=163.0,
dtype='dBm', ndata=3584, bw=3690, processing='peak', antuid=0, thresh=-100, minimum=-147.5), 
SpecData(type=68, thread_id=371, description='PMEC 2021 (Faixa 8 de 10). @ -80dBm, 100kHz.', 
start_mega=2690.0, stop_mega=1002898.999, dtype='dBm', ndata=5376, bw=73828, 
processing='peak', antuid=0, thresh=-100, minimum=-147.5), SpecData(type=68, thread_id=351, 
description='PMEC 2021 (Faixa 6 de 10). @ -80dBm, 100kHz.', start_mega=1001388.999, 
stop_mega=1001428.999, dtype='dBm', ndata=1280, bw=73828, processing='peak', antuid=0, 
thresh=-100, minimum=-147.5), SpecData(type=68, thread_id=331, description='PMEC 2021 (Faixa 
4 de 10). @ -80dBm, 100kHz.', start_mega=400.0, stop_mega=410.0, dtype='dBm', ndata=256, 
bw=73828, processing='peak', antuid=0, thresh=-100, minimum=-147.5), SpecData(type=68, 
thread_id=361, description='PMEC 2021 (Faixa 7 de 10). @ -80dBm, 100kHz.', start_mega=1530.0,
stop_mega=1001648.999, dtype='dBm', ndata=3072, bw=73828, processing='peak', antuid=0, 
thresh=-100, minimum=-147.5)]
}
fluxo = compressed['spectrum'] ; fluxos
(#20) [SpecData(type=67, thread_id=300, description='PMEC 2021 (Faixa 1 de 10).', start_mega=105.0, stop_mega=140.0, dtype='dBm', ndata=3584, bw=18457, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5),SpecData(type=67, thread_id=310, description='PMEC 2021 (Faixa 2 de 10).', start_mega=155.0, stop_mega=165.0, dtype='dBm', ndata=1024, bw=18457, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5),SpecData(type=67, thread_id=100, description='PRD 2021 (Faixa principal 1 de 4).', start_mega=50.0, stop_mega=90.0, dtype='dBμV/m', ndata=1024, bw=73828, processing='peak', antuid=0, thresh=-90.5, minimum=-90.5),SpecData(type=67, thread_id=110, description='PRD 2021 (Faixa principal 2 de 4).', start_mega=70.0, stop_mega=110.0, dtype='dBμV/m', ndata=1024, bw=73828, processing='peak', antuid=0, thresh=-67.5, minimum=-67.5),SpecData(type=67, thread_id=120, description='PRD 2021 (Faixa principal 3 de 4).', start_mega=170.0, stop_mega=220.0, dtype='dBμV/m', ndata=1280, bw=73828, processing='peak', antuid=0, thresh=-37.5, minimum=-37.5),SpecData(type=67, thread_id=130, description='PRD 2021 (Faixa principal 4 de 4).', start_mega=470.0, stop_mega=700.0, dtype='dBμV/m', ndata=5888, bw=73828, processing='peak', antuid=0, thresh=-41.5, minimum=-41.5),SpecData(type=67, thread_id=200, description='PMEF 2021 (Faixa 1 de 6).', start_mega=700.0, stop_mega=960.0, dtype='dBm', ndata=6656, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5),SpecData(type=67, thread_id=210, description='PMEF 2021 (Faixa 2 de 6).', start_mega=1710.0, stop_mega=1980.0, dtype='dBm', ndata=6912, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5),SpecData(type=67, thread_id=220, description='PMEF 2021 (Faixa 3 de 6).', start_mega=2100.0, stop_mega=1002168.999, dtype='dBm', ndata=1792, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5),SpecData(type=67, thread_id=230, description='PMEF 2021 (Faixa 4 de 6).', start_mega=2290.0, stop_mega=2390.0, dtype='dBm', ndata=2560, bw=73828, processing='peak', antuid=0, thresh=-147.5, minimum=-147.5)...]
fluxo = fluxos[0]
fluxo.matrix().iloc[:5, [0, 1, 2, -3, -2, -1]]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 105.000000 105.009768 105.019537 139.980463 139.990232 140.000000
Time
2021-06-26 12:01:00.731405 -112.5 -117.0 -111.0 -106.0 -105.0 -99.5
2021-06-26 12:02:00.731287 -111.5 -115.5 -116.5 -103.0 -101.5 -101.0
2021-06-26 12:03:00.731107 -116.5 -113.0 -113.5 -105.5 -103.5 -105.0
2021-06-26 12:04:00.731401 -111.5 -112.0 -110.0 -109.5 -114.5 -106.5
2021-06-26 12:05:00.731532 -113.0 -115.0 -117.5 -101.5 -101.0 -99.5
print(len(fluxo))
8715

CRFS Bin - Versão 4

file = r'binfiles\v4\rfeye002292_210208_T202215_CRFSBINv.4.bin'
blocks = parse_bin(file)
print(blocks)
{
    'filename': 'rfeye002292_210208_T202215_CRFSBINv.4.bin',
    'file_version': 22,
    'string': 'CRFS DATA FILE V022',
    'hostname': 'rfeye002292',
    'method': 'Script_CRFSBINv4',
    'unit_info': 'Stationary',
    'file_number': 0,
    'identifier': 'LOGGER_VERSION',
    'group_id': 0,
    'text': 'ClearWrite. Peak.',
    'gps': GPS Data - Median of Coordinates: -14.81416:-39.03184 Altitude: 15.50 #Satellites:
11.0,
    'spectrum': [SpecData(type=63, thread_id=20, description='Peak', start_mega=76.0, 
stop_mega=108.0, dtype='dBm', ndata=8192, bw=3, processing='peak', antuid=0, thresh=-147.5, 
minimum=-147.5), SpecData(type=63, thread_id=30, description='Peak', start_mega=70.0, 
stop_mega=110.0, dtype='dBm', ndata=1024, bw=39, processing='peak', antuid=0, thresh=-147.5, 
minimum=-147.5), SpecData(type=63, thread_id=10, description='Peak', start_mega=105.0, 
stop_mega=140.0, dtype='dBm', ndata=3584, bw=9, processing='peak', antuid=0, thresh=-147.5, 
minimum=-147.5), SpecData(type=63, thread_id=11, description='Peak', start_mega=105.0, 
stop_mega=140.0, dtype='dBm', ndata=3584, bw=9, processing='average', antuid=0, 
thresh=-147.5, minimum=-147.5), SpecData(type=63, thread_id=12, description='Peak', 
start_mega=105.0, stop_mega=140.0, dtype='dBm', ndata=3584, bw=9, processing='peak', 
antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=63, thread_id=13, description='Peak',
start_mega=105.0, stop_mega=140.0, dtype='dBm', ndata=3584, bw=9, processing='average', 
antuid=0, thresh=-147.5, minimum=-147.5), SpecData(type=64, thread_id=14, description='Peak',
start_mega=105.0, stop_mega=140.0, dtype='dBm', ndata=3584, bw=48, processing='average', 
antuid=0, thresh=-90, minimum=-147.5), SpecData(type=65, thread_id=15, start_mega=105.0, 
stop_mega=140.0, dtype='dBm', ndata=3584, processing='peak', antuid=0, thresh=-90), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=51, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=47, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=46, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=59, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=44, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=62, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=50, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=49, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=43, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=52, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=54, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=42, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=37, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=53, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=45, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=55, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=58, processing='average', antuid=0, thresh=-90, minimum=-147.5), 
SpecData(type=64, thread_id=14, description='Peak', start_mega=105.0, stop_mega=140.0, 
dtype='dBm', ndata=3584, bw=39, processing='average', antuid=0, thresh=-90, minimum=-147.5)]
}
blocks['spectrum'][0].matrix().iloc[:5, [0, 1, 2, -3, -2, -1]]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 76.000000 76.003907 76.007813 107.992187 107.996093 108.000000
Time
2021-02-08 20:22:15.500658 -110.0 -100.5 -99.0 -103.0 -99.0 -96.0
2021-02-08 20:22:16.142770 -105.5 -100.0 -97.5 -94.5 -95.0 -98.0
2021-02-08 20:22:16.500750 -104.0 -102.5 -105.5 -95.5 -98.5 -93.0
2021-02-08 20:22:17.132990 -105.0 -107.0 -103.0 -99.5 -99.5 -102.5
2021-02-08 20:22:17.501352 -97.5 -101.5 -97.0 -104.5 -102.0 -99.5

CRFS Bin - Versão 3

file = r'binfiles\v3\rfeye002292_210208_T203238_CRFSBINv.3.bin'
blocks = parse_bin(file)
print(blocks)
{
    'filename': 'rfeye002292_210208_T203238_CRFSBINv.3.bin',
    'file_version': 21,
    'string': 'CRFS DATA FILE V021',
    'hostname': 'rfeye002292',
    'method': 'Script_CRFSBINv3',
    'unit_info': 'Stationary',
    'file_number': 0,
    'identifier': 'LOGGER_VERSION',
    'gps': GPS Data - Median of Coordinates: -14.81416:-39.03184 Altitude: 15.50 #Satellites:
11.0,
    'spectrum': [SpecData(type=60, thread_id=10, start_mega=105, stop_mega=140, ndata=3584, 
nloops=1, processing='peak', antuid=0, minimum=-147.5), SpecData(type=60, thread_id=11, 
start_mega=105, stop_mega=140, ndata=3584, nloops=1, processing='average', antuid=0, 
minimum=-147.5), SpecData(type=60, thread_id=20, start_mega=76, stop_mega=108, ndata=8192, 
nloops=1, processing='peak', antuid=0, minimum=-147.5), SpecData(type=60, thread_id=30, 
start_mega=70, stop_mega=110, ndata=1024, nloops=4, processing='peak', antuid=0, 
minimum=-147.5), SpecData(type=60, thread_id=12, start_mega=105, stop_mega=140, ndata=3584, 
nloops=1, processing='peak', antuid=0, minimum=-147.5), SpecData(type=60, thread_id=13, 
start_mega=105, stop_mega=140, ndata=3584, nloops=1, processing='average', antuid=0, 
minimum=-147.5), SpecData(type=61, thread_id=14, start_mega=105, stop_mega=140, ndata=3584, 
nloops=1, processing='average', antuid=0, thresh=-90, minimum=-147.5), SpecData(type=62, 
thread_id=15, start_mega=105000000, stop_mega=140000000, sampling=9, ndata=3584, antuid=0, 
thresh=-90), SpecData(type=62, thread_id=15, start_mega=105000000, stop_mega=140000000, 
sampling=10, ndata=3584, antuid=0, thresh=-90)]
}
blocks['spectrum'][0].matrix().iloc[:5, [0, 1, 2, -3, -2, -1]]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 105.000000 105.009768 105.019537 139.980463 139.990232 140.000000
Time
2021-02-08 20:32:39.548000 -76.5 -76.0 -76.5 -94.5 -91.0 -90.0
2021-02-08 20:32:40.133600 -79.5 -80.5 -79.5 -99.0 -94.5 -92.5
2021-02-08 20:32:41.858000 -69.0 -69.0 -69.0 -97.5 -92.5 -90.0
2021-02-08 20:32:42.137500 -70.5 -71.0 -71.5 -97.0 -98.0 -94.5
2021-02-08 20:32:43.716000 -71.0 -69.5 -70.0 -97.0 -89.0 -87.0

CRFS Bin Versão 2

from rfpye.parser import parse_bin
file = r'binfiles\v2\rfeye002092_210208_T203131_CRFSBINv.2.bin'
blocks = parse_bin(file)
print(blocks)
{
    'filename': 'rfeye002092_210208_T203131_CRFSBINv.2.bin',
    'file_version': 21,
    'string': 'CRFS DATA FILE V021',
    'size': 24,
    'text': 'ClearWrite. Peak.',
    'textlen': 20,
    'thread_id': 30,
    'type': 3,
    'gps': GPS Data - Median of Coordinates: -14.81416:-39.03184 Altitude: 15.50 #Satellites:
11.0,
    'spectrum': [SpecData(type=4, thread_id=20, start_mega=76, stop_mega=108, ndata=8192, 
processing='Peak'), SpecData(type=4, thread_id=30, start_mega=70, stop_mega=110, ndata=1024, 
processing='Peak'), SpecData(type=4, thread_id=10, start_mega=105, stop_mega=140, ndata=3584,
processing='Peak'), SpecData(type=4, thread_id=11, start_mega=105, stop_mega=140, ndata=3584,
processing='Average'), SpecData(type=4, thread_id=12, start_mega=105, stop_mega=140, 
ndata=3584, processing='Peak'), SpecData(type=4, thread_id=13, start_mega=105, stop_mega=140,
ndata=3584, processing='Average'), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=203, ndata=3584, thresh=-90), SpecData(type=8, thread_id=15, 
start_mega=105, stop_mega=140, sampling=9, namal=10, ndata=3584, thresh=-90), 
SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=204, ndata=3584, 
thresh=-90), SpecData(type=8, thread_id=15, start_mega=105, stop_mega=140, sampling=10, 
namal=10, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=221, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, 
start_mega=105, stop_mega=140, namal=206, ndata=3584, thresh=-90), SpecData(type=7, 
thread_id=14, start_mega=105, stop_mega=140, namal=207, ndata=3584, thresh=-90), 
SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=213, ndata=3584, 
thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=209, 
ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, 
namal=226, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=212, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, 
start_mega=105, stop_mega=140, namal=215, ndata=3584, thresh=-90), SpecData(type=7, 
thread_id=14, start_mega=105, stop_mega=140, namal=208, ndata=3584, thresh=-90), 
SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=210, ndata=3584, 
thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=201, 
ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, 
namal=219, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=202, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, 
start_mega=105, stop_mega=140, namal=222, ndata=3584, thresh=-90), SpecData(type=7, 
thread_id=14, start_mega=105, stop_mega=140, namal=220, ndata=3584, thresh=-90), 
SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=216, ndata=3584, 
thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=224, 
ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, 
namal=229, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=218, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, 
start_mega=105, stop_mega=140, namal=211, ndata=3584, thresh=-90), SpecData(type=7, 
thread_id=14, start_mega=105, stop_mega=140, namal=223, ndata=3584, thresh=-90), 
SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=205, ndata=3584, 
thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, namal=214, 
ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, stop_mega=140, 
namal=228, ndata=3584, thresh=-90), SpecData(type=7, thread_id=14, start_mega=105, 
stop_mega=140, namal=230, ndata=3584, thresh=-90)]
}
blocks['spectrum'][0].matrix().iloc[:5, [0, 1, 2, -3, -2, -1]]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 76.000000 76.003907 76.007813 107.992187 107.996093 108.000000
Time
0 10.5 11.5 13.0 31.0 26.0 22.5
1 18.0 19.5 16.5 27.5 29.5 25.5
2 11.5 5.0 8.0 30.5 28.5 30.0
3 17.5 15.5 11.0 33.5 29.5 30.0
4 13.0 12.5 14.0 30.0 29.5 28.5

Fluxo de Ocupação

from rfpye.parser import parse_bin
file = r'binfiles\occ\rfeye002090-VCP_FM_occ15min_191221_085803.bin'
blocks = parse_bin(file)
print(blocks)
{
    'filename': 'rfeye002090-VCP_FM_occ15min_191221_085803.bin',
    'file_version': 22,
    'string': 'CRFS DATA FILE V022',
    'hostname': 'rfeye002090-VCP',
    'method': 'CRFS default method',
    'unit_info': 'RFeye002090',
    'file_number': 0,
    'identifier': 'LOGGER_VERSION',
    'group_id': 0,
    'text': 'Ocupacao em 15 minutos na faixa FM',
    'gps': GPS Data - Median of Coordinates: -1.00000:-1.00000 Altitude: -1.00 #Satellites: 
0.0,
    'spectrum': [SpecData(type=65, thread_id=121, start_mega=80.0, stop_mega=110.0, 
dtype='dBm', ndata=1536, processing='peak', antuid=0, thresh=-60)]
}
blocks['spectrum'][-1].matrix().iloc[:5, 1003:1010]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
Frequencies 99.602606 99.622150 99.641694 99.661238 99.680782 99.700326 99.719870
Time
2019-12-21 09:00:01.367337 17.5 36.0 53.5 62.5 76.5 80.0 72.5
2019-12-21 09:15:01.357259 15.0 29.5 48.0 61.0 76.5 78.5 71.0
2019-12-21 09:30:01.357357 16.0 28.5 46.5 61.0 76.5 77.5 70.0
2019-12-21 09:45:01.357273 15.0 33.0 49.0 66.0 76.0 78.0 70.0
2019-12-21 10:00:01.419225 15.5 35.0 50.0 62.0 74.0 77.0 67.5

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rfpye-0.3.0.tar.gz (172.3 kB view hashes)

Uploaded Source

Built Distribution

rfpye-0.3.0-cp39-cp39-win_amd64.whl (208.8 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page