Skip to main content

Scraping module for Korean Bible University

Project description

biblebot-scraper

biblebot-scraper is a python module for scrapping web pages of the Korean Bible University and is distributed under the MIT license.

The project was started in 2018 by Kyungmin Lee.

Installation

The recommended way to install the biblebot module is to simply use pip

$ pip install biblebot

dependencies

  • Python>=3.7
  • aiohttp>=3.6.2, <4
  • beautifulsoup4>=4.8.1, <5
  • lxml>=4.4.2, <5
  • Pillow>=6.2.1, <7

Getting started

Intranet example:

import asyncio
from pprint import pprint

from biblebot import Intranet, AlertError


async def main():
    account = ('your-id', 'your-password')  # fill

    async with Intranet() as intranet:
        try:
            await intranet.login(*account)
            print('login success')

            sid = await intranet.get_student_id()
            print(f'student id: {sid}')

            timetable = await intranet.get_timetable()
            pprint(timetable)

        except AlertError as e:
            print(f'alert message: {e}')

if __name__ == '__main__':
    asyncio.run(main())

output example:

login success
student id: 201504021
{'selectable': ['20193', '20192', '20191', '20182', '20181', '20152', '20151'],
 'selected': '20192',
 'tbody': [[['무선및모바일통신', '정보2실', '13:30', '14:45'],
            ['데이터마이닝과통계', '정보2실', '14:55', '16:10']],
           [['데이터마이닝과통계', '정보2실', '10:25', '11:40'],
            ['네트워크프로그래밍', '정보2실', '13:30', '14:45']],
           [['세계문명과기독교Ⅱ', '천마홀', '09:00', '10:15'],
            ['전도훈련Ⅵ', '모리아관 301호', '13:30', '15:20'],
            ['미래설계상담', '정보3실', '15:30', '16:20']],
           [['네트워크프로그래밍', '정보2실', '09:00', '10:15'],
            ['무선및모바일통신', '정보2실', '13:30', '14:45']],
           []],
 'thead': ['월(MON)', '화(TUE)', '수(WED)', '목(THU)', '금(FRI)']}

LMS example:

import asyncio
from pprint import pprint

from biblebot import Lms, AlertError


async def main():
    account = ('your-id', 'your-password')  # fill

    async with Lms() as lms:
        try:
            await lms.login(*account)
            lectures = await lms.get_lectures()

            attendances = await asyncio.gather(
                *[lms.get_attendance(code) for code in lectures['data'].values()]
            )
            pprint(attendances)

        except AlertError as e:
            print(f'alert message: {e}')

if __name__ == '__main__':
    asyncio.run(main())

output example:

[{'extra': {'강좌명': '네트워크프로그래밍[A]',
            '강좌번호': 'IC111',
            '분반번호': 'A',
            '이름': '이경민 (201504021)'},
  'summary': {'기타': '0', '일반결석': '0', '지각': '2', '출석': '29'},
  'tbody': [['2019-08-27', '13:30', '○', '', '', ''],
            ['2019-08-29', '09:00', '○', '', '', ''],
            ['2019-09-03', '13:30', '○', '', '', ''],
            ['2019-09-05', '09:00', '○', '', '', ''],
            ['2019-09-10', '13:30', '○', '', '', ''],
            ['2019-09-17', '13:30', '○', '', '', ''],
            ['2019-09-19', '09:00', '○', '', '', ''],
            ['2019-09-24', '13:30', '○', '', '', ''],
            ['2019-09-26', '09:00', '○', '', '', ''],
            ['2019-10-01', '13:30', '○', '', '', ''],
            ['2019-10-08', '13:30', '○', '', '', ''],
            ['2019-10-10', '09:00', '○', '', '', ''],
            ['2019-10-15', '13:30', '○', '', '', ''],
            ['2019-10-17', '09:00', '○', '', '', ''],
            ['2019-10-22', '13:30', '○', '', '', ''],
            ['2019-10-24', '09:00', '○', '', '', ''],
            ['2019-10-29', '13:30', '○', '', '', ''],
            ['2019-10-31', '09:00', '○', '', '', ''],
            ['2019-11-05', '13:30', '○', '', '', ''],
            ['2019-11-06', '10:25', '○', '', '', ''],
            ['2019-11-07', '09:00', '○', '', '', ''],
            ['2019-11-12', '13:30', '○', '', '', ''],
            ['2019-11-14', '09:00', '○', '', '', ''],
            ['2019-11-19', '13:30', '○', '', '', ''],
            ['2019-11-21', '09:00', '○', '', '', ''],
            ['2019-11-26', '13:30', '○', '', '', ''],
            ['2019-11-28', '09:00', '', '', '○', ''],
            ['2019-12-03', '13:30', '○', '', '', ''],
            ['2019-12-05', '09:00', '', '', '○', ''],
            ['2019-12-10', '13:30', '○', '', '', ''],
            ['2019-12-12', '09:00', '○', '', '', '']],
  'thead': ['출결 날짜', '시간', '출석', '일반결석', '지각', '기타']},

...

 {'extra': {'강좌명': '무선및모바일통신[A]',
            '강좌번호': 'IC149',
            '분반번호': 'A',
            '이름': '이경민 (201504021)'},
  'summary': {'기타': '0', '일반결석': '0', '지각': '2', '출석': '29'},
  'tbody': [['2019-08-26', '13:30', '○', '', '', ''],
            ['2019-08-29', '13:30', '○', '', '', ''],
            ['2019-09-02', '13:30', '○', '', '', ''],
            ['2019-09-05', '13:30', '○', '', '', ''],
            ['2019-09-09', '13:30', '○', '', '', ''],
            ['2019-09-12', '13:30', '', '', '', ''],
            ['2019-09-16', '13:30', '○', '', '', ''],
            ['2019-09-19', '13:30', '○', '', '', ''],
            ['2019-09-23', '13:30', '○', '', '', ''],
            ['2019-09-26', '13:30', '○', '', '', ''],
            ['2019-09-30', '13:30', '○', '', '', ''],
            ['2019-10-03', '13:30', '', '', '', ''],
            ['2019-10-07', '13:30', '○', '', '', ''],
            ['2019-10-10', '13:30', '○', '', '', ''],
            ['2019-10-14', '13:30', '○', '', '', ''],
            ['2019-10-17', '13:30', '○', '', '', ''],
            ['2019-10-21', '13:30', '○', '', '', ''],
            ['2019-10-24', '13:30', '○', '', '', ''],
            ['2019-10-24', '14:55', '○', '', '', ''],
            ['2019-10-28', '13:30', '○', '', '', ''],
            ['2019-10-31', '13:30', '○', '', '', ''],
            ['2019-11-04', '13:30', '', '', '○', ''],
            ['2019-11-07', '13:30', '○', '', '', ''],
            ['2019-11-11', '13:30', '○', '', '', ''],
            ['2019-11-14', '13:30', '○', '', '', ''],
            ['2019-11-18', '13:30', '', '', '○', ''],
            ['2019-11-21', '13:30', '○', '', '', ''],
            ['2019-11-25', '13:30', '○', '', '', ''],
            ['2019-11-28', '13:30', '○', '', '', ''],
            ['2019-12-02', '13:30', '○', '', '', ''],
            ['2019-12-05', '13:30', '○', '', '', ''],
            ['2019-12-09', '13:30', '○', '', '', ''],
            ['2019-12-12', '13:30', '○', '', '', '']],
  'thead': ['출결 날짜', '시간', '출석', '일반결석', '지각', '기타']}]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for biblebot, version 1.1.3
Filename, size File type Python version Upload date Hashes
Filename, size biblebot-1.1.3-py3-none-any.whl (17.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size biblebot-1.1.3.tar.gz (15.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page