Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Acquisition, Search, References:
Introduction to Data Acquisition |…
Data Acquisition
Public Data
🤖 Web scraping tools:
References:
Rules of scraping:
- Check the website's T&Cs
- Do NOT spam the website with a ton of requests
- Make one request to one webpage per second
-
Application Programming Interfaces (APIs):
- APIs are built around the HTTP Request/Response Cycle
- Many APIs request that users sign up and obtain an API key that uniquely identifies them and records of all a user’s requests
-
-
Ethics — With all data acquisition methods, there are ethical considerations that are important to make:
- Who owns the data uploaded to a website by users?
- When and how should users of services be notified that data about them is being acquired?
- What kinds of data should be restricted from being acquired about users?
- How can users protect their privacy and know when it has been breached?
-
-
-
-
Data acquisition should be:
- Fair
- Transparent
- Respectful
-