Please enable JavaScript.

Coggle requires JavaScript to display documents.

FinLab (第一章 Finlab財經實驗室架設 (單元二打造專屬python實驗室II (:no_entry: Create new…

- - - - anaconda
      - jupyterlab
  - - - conda create -n finlab python=3.6
      - conda activate finlab
      - conda install --file <spec file>
      - python -m ipykernal install --user --name finlab
      - install
        
        ipywidgets (nodejs)
        
        <finlab> conda install -c conda -forge ipywidgets
        
        <base> jupyter labextension install jupyter-widgets/jupyterlab-manager
        
        install nodejs 5+
        
        talib
        
        download from website ( window x64)
        
        activate <finlb> pip install whl.
    - - powershell -Command "(New-Object Net.WebClient).DownloadFile('https://drive.google.com/uc?export=download&id=1ZLo49sqFyKK3OaGj4Y6RkYxqYXNh29qr', 'setup_env.bat')"
      - echo Y | setup_env.bat
  - - - 歷史資料
        
        https://drive.google.com/file/d/1c8sJkBuncrrfpA2hHZH8A6RS96tQQsmb/view?usp=sharing
      - 課程
        
        Finlab API
        
        finlab.zip
        
        課程範例
        
        notebooks.zip
    - - requests
        
        把response from json to csv
        
        ✍pd.reqd_csv(is.StringIO(res.text.replace("=","")),header=1,index_col="時間")
      - dropna
        
        df=df.dropna(how="all",axis=0).dropna(how="all",axis=1)
      - date processing
        
        date=datetime.date(2019,1,2)
        
        2019-1-2
        
        datestr=date.strftime("%Y%m%d")
        
        20190102
      - index 加日期
        
        df.index= pd.to_datetime(datastr+" "+df.index)
      - 取值 df[2019-1-2 9:00 :2019-1-2 11:00]
      - 去除逗號，把str轉float
        
        df.applymap(lambda s:float(str(s).replace(",","")))
  - - - 日
        
        開高低收
        
        crawl_price
        
        上市
        
        price_twe
        
        'https://www.tpex.org.tw/web/stock/aftertrading/daily_close_quotes/stk_quote_download.php?l=zh-tw&d='+datestr+'&s=0,asc,0'
        
        上櫃
        
        price_otc
        
        https://www.twse.com.tw/exchangeReport/MI_INDEX?response=csv&date=20190102&type=ALLBUT0999
        
        三大法人買賣超
        
        crawl_bargin
        
        上市
        
        https://www.twse.com.tw/fund/T86?response=csv&date='+datestr+'&selectType=ALLBUT0999
        
        上櫃
        
        https://www.tpex.org.tw/web/stock/3insti/daily_trade/3itrade_hedge_result.php?l=zh-tw&o=csv&se=EW&t=D&d='+datestr+'&s=0,asc
        
        本益比，殖利率，股價淨值比
        
        crawl_pe
        
        上市
        
        https://www.twse.com.tw/exchangeReport/BWIBBU_d?response=csv&date='+datestr+'&selectType=ALL
        
        上櫃
        
        https://www.tpex.org.tw/web/stock/aftertrading/peratio_analysis/pera_result.php?l=zh-tw&o=csv&charset=UTF-8&d='+datestr+'&c=&s=0,asc
      - 月
        
        營業收入統計表
        
        crawl_monthly_report
        
        上市
        
        https://mops.twse.com.tw/nas/t21/sii/t21sc03_108_11.html
        
        上櫃
        
        https://mops.twse.com.tw/nas/t21/otc/t21sc03_108_11.html
      - 季
        
        資產負債表
        損益表
        現金流量表
        權益變動表
        
        crawl_finance_statement_by_date (zip)
        
        https://mops.twse.com.tw/server-java/FileDownLoad?step=9&fileName=tifrs-2019Q3.zip&filePath=/home/html/nas/ifrs/2019/
      - 一次性
        
        除權息
        
        divide_ratio
        
        crawl_twse_divide_ratio
        
        crawl_otc_divide_ratio
        
        增減資
        
        cap_reduction
        
        crawl_twse_cap_reduction
        
        crawl_otc_cap_reduction
    - - 天
        
        date_range(startdate, enddate)
      - 月
        
        month_range(startdate, enddate)
      - 季
        
        season_range(datetime.date(2018,1,2),datetime.date(2019,1,10))
        
        財報截止時間 3/31 , 5/15 , 8/14 , 11/14
    - - 爬蟲function 型態
        
        crawl_price(datetime.date(2020,1,2))
        
        雙層index
        
        index1 : stock_id (1101 台泥)
        
        index2 : date (2019-01-02)
      - ✍pivot ()
        
        資料重塑
      - commit()
  - - - 將index 轉雙index
        
        df=df.set_index([stock_id,date])
      - def crawl_benchmark(date)
      - 例外處理
        
        return pd.Dataframe()
        print("error")
    - - 需傳入日期
        
        取得database裡面的起始跟結束日期
        
        start_date=table_date_range("price")[1]
        
        今日日期
        
        end_date= datetime.datetime.now()
        
        枚舉中間這段時間
        
        dates=date_range(start_date,end_date)
        
        下載並合併資料庫
        
        ⭐update_table (table_name,crawl_function,dates)
        
        要爬取的
        
        table
        
        table_name="price"
        
        function
        
        crawl_function="crawl_price"
      - 不須傳入日期
        
        偵測function 是否需傳入日期
        
        ✍
        import inspect
        sig=inspect.signature(function)
        
        to_pickle(df)
      - commit()
    - - from finlab.date import Data
        data=Data()
        df=date.get("收盤價")