Please enable JavaScript.

Coggle requires JavaScript to display documents.

ПИТЕР Я УЖЕ ТАМ (What is RR :!?: (A study is reproducible, if (the…

- - - - instruments
      - analytical methods
      - investigators
      - data
      - laboratories
  - - - code and data
  - - - where likelihood to replicate fully is time sensitive
- - - - literate program
        
        weaved
        
        to produce human readable doc
        
        tangled
        
        to produce machine readable doc
      - a useful way to put text, data, code and output all in one doc
        incld. graphical results
        
        needs
        
        a documentation language
        
        a programming language
        
        Knitr supports a variety
        of documentation languages
        
        knitr is a programming language specific tool
      - Pros :+1:
        
        logical order of text & code
        
        code is live
        
        data, results are automatically updated
        
        to reflect external changes
      - cons :-1:
        
        difficult to read, a lot of codes
        
        substantially slow down processing of documents
      - evolution
        
        HTML :arrow_left:
        
        Latex :arrow_left:
        
        Sweave :arrow_left:
        
        restructured text :arrow_left:
        
        Markdown
        
        idea origin: Don Knuth
  - - - Galaxy
      - Taverna
      - Sumatra
    - - tracking of chronology & origin of research objects e.g.
        data, source code, figures & results
  - - - Github
      - Google Code
  - - - that represent sequences of commands & calculations as an interactive worksheet with pretty printing and integrated displays,
        decoupling content (the data, calculations) from representation (PDF, HTML), so that the same research content can be presented in multiple ways
      - Rstudio with knit R, IPython
- - - - results can't be reproduced by independent researcher
    - - results cannot seem to be reproduced by independent researchers
    - - results could be reproduced with extreme efforts by independent researcher
    - - results could be reproduced with considerable efforts by independent researcher
    - - can be easily reproduced with at most 15 minutes, require proprietary source packages e.g. SAS
    - - can be easily reproduced with at most 15 minutes, require only standard freely available tools e.g. R, Python
    - - Reviewable Research
        
        descriptions of research methods can be independently assessed
        
        e.g. traditional peer reviewed
      - Replicable research
        
        tools made available, allow one to duplicate results of research
      - Confirmable Research
        
        main conclusion of research can be attained independently
        w/o use of software provided by author
      - Auditable Research
        
        sufficient records (data & software included) archived,
        to defend research in future,
        in case of differences of independent findings
      - Open or Reproducible Research
        
        Auditable research made openly available
        
        comprised well documented and fully open, publicly avaliable code and data
        
        this allows
        
        a) fully audit of computational procedure
        
        b) replicate & independently reproduce results of research
        
        c) extend results, apply methods to new probles
    - - Better
        
        Use version control
        
        keeps track of changes, fix bugs,
        improve project management
      - Good
        
        Use code
        
        minimize pointing and clicking (RStudio). Mention availability of code
      - Best
        
        e.g. RStudio, Git & Github combo
        
        Use embedded narrative and code to explicitly link code, text and data
        
        save your time, reviewers time, improve code
  - - - an analysis can be reproducible and still be wrong
  - - - share in open repositories
        
        data, software, workflows,
        details of computational environment
    - - persistent links should appear in published doc
      - include permanent identifier for data, code, and digital artifacts
    - - citation, standard practice
    - - adequately document digital scholarly artifacts
    - - should, by Journals
    - - to publish digital scholarly objects
- - - - replication
        
        the confirmation of results & conclusions from one study obtained independently in another
        
        Full replication is scientific gold standard
      - sole publication not reproducible
      - Reproducibility
        
        minimum standard
        
        for assessing the value of scientific claims
        
        particularly, when full independent replication of a study is not feasible
  - - - completely independent reproduction
        
        w/o access to codes
        
        based on only text description
    - - reproduction of the original results using the same tools
        
        original author/same machine
        
        someone same lab/same machine
        
        someone in different lab
  - - - reproducibility
        
        obtain the same results
        using a different program
      - repeatability
        
        re-run code and get different results
      - repeatability & reproducibility
        
        re-run code and get same results
- - - - supports variety documentation language
        
        HTML
        
        RMarkdown
        
        LaTeX
    - - data pre-processing
      - short/medium length technical doc
      - tutorials
      - reports - periodical
    - - complex time-consuming computations
      - long research articles
      - documents that require precise formatting
  - - - Rights framework for dataset
        
        standards for exchanging data
        
        protocols
      - tools for researchers, developers
        
        software tools
      - dataset repositories
        
        servers
        
        website
      - other necessary information
    - - Decide to do it at the start
      - keep track of things i.e. version control
      - use software whose operations can be coded
      - don't save output
      - save data in non-proprietary formats
    - - Begin with the final product in mind
      - use literate programming
      - keep history of changes via code versioning & sharing
      - Get basic statistics right
      - set stringent cutoffs, correct p-values for multiple testing
      - Be critical
        
        control validation
        
        do sanity checks
        
        consider batch effects
        
        visualize
        
        use random controls
      - Follow reporting guidelines
- - - - people are interested in the science, high-level and not the experimental setup
    - - pseudo-code, a great way of communicating ideas quickly and clearly, - not understand the subtle implementation details
    - - any limitations of methods/proofs will be obvious to careful reader
    - - others won't understand how the code actually work or doesn't actually work
  - - - trends
        
        scientific research becoming more computational
        
        Scientific computing will become dramatically transparent
      - scientific evidence is strengthened when important findings are replicated by independent parties
      - the success and credibility of Science should depend on willingness of scientists towards independent testing and replication
    - - open research
        
        need to reward publication of research output along the process
        
        not just based on article is published
        
        scientific knowledge of all kinds should be openly shared, as early as practical in the discovery process
- - - - can determine model performance based on comparison of new analysis with reproduced old analysis
      - can properly justify adding creative sophistication or complexity to analysis
      - can avoid repeating mistakes
    - - data science is about collaboration
      - share work, and data product that allows others to replicate, and produce the same results
      - team will able to capitalize on its collective knowledge, and advances within the team
    - - Establish audit trail & explain to satisfy regulatory
        concerns and public demand
        
        especially if applications and
        decision-making impact users