Please enable JavaScript.

Coggle requires JavaScript to display documents.

Chapter 14: Conclusion (Fundamental Concepts of Data Science (These…

- - - - Types:
        
        (1) General concepts about how data science fits in the organization and the competitive landscape, including ways to attract, structure, ways for thinking about how data science lead to competitive advantage, ways to be sustained, and tactical principles for doing well with data science.
        
        (2) General ways of thinking data-analytically, the concepts include the data mining process, the collection of different high-level data science tasks, as well as principles.
        
        Keep in mind the problem to be solved and the use scenario throughout the data mining process.
        
        Data should be considered an asset, therefore think carefully about what investments we should make to get the best leverage from our asset.
        
        The expected value framework can help us to structure business problems so we can see the component data mining problems as well as the connective tissue of costs, benefits, and constraints imposed by the business environment
        
        Generalization and overfitting: we want patterns that generalize to data we have not yet seen.
        
        Applying data science to a well-structured problem versus exploratory data mining require different levels of effort in different stages of the data mining process.
        
        (3) General concepts for actually extracting knowledge from data:
        
        Identifying informative attributes — those that correlate with or give us information about an unknown quantity of interest.
        
        Fitting a numeric function model to data by choosing an objective and finding a set of parameters based on that objective.
        
        Controlling complexity is necessary to find a good trade-off between generalization and overfitting.
        
        Calculating similarity between objects described by data.
- - - - Recently there has been a marked shift in consumer online activity from traditional computers to a wide variety of mobile devices.
        
        Data-analytic thinker might notice that mobile devices provide a new sort of data from which little leverage has yet been obtained. In particular, mobile devices are associated with data on their location.
        
        Use of that data:
        
        If we want to get beyond exploratory data analysis, we need to think in terms of some concrete business problem.
        
        Advertisers face the problem that in this new world, we see a variety of different devices and a particular consumer’s behavior may be fragmented across several.
        
        Desktop
        
        In the desktop world, once the advertisers identify a good prospect, perhaps via a cookie in a particular consumer’s browser or a device ID.
        
        Mobile Devices
        
        In the mobile ecosystem, this consumer’s activity is fragmented across devices.
        
        One possibility is to use the location data to winnow the space of possible other devices that could belong to this prospect.
        
        When working through our data-understanding phase, we need to decide how exactly we will represent devices and their locations.
        
        When mining data on documents, we often ignore much of the structure of the text, such as its sequence. For many problems we can simply treat each document as a collection of words from a potentially large vocabulary.
        
        TFIDF
        
        If we are looking to try to find other instances of the same user, we might also profitably apply the ideas of TFIDF for text to our locations.
        
        Example:
        
        WiFi locations that are very popular are unlikely to be so informative in a similarity calculation focused on finding the same user on different devices. Such a location would get a low IDF score.
        
        TFIDF on location would magnify the importance of these locations in a similarity calculation.
        
        if our device profile is a TFIDF representation based on our bag of locations, as with using similarity over the TFIDF for our search we might look for the devices most similar to the one that we had identified as a good prospect.
        
        1 more item...
- - - - The structure imposed by the process emphasizes the interaction of humans early, to ensure that the application of data science methods are focused on the right tasks.
        
        Examining the data mining process also reveals that task selection and specification is not the only place where human interaction is critical.
        
        Human interaction is also critical in the evaluation stage of the process.
        
        Only humans can tell what is the best objective criterion to optimize for a particular problem. This involves substantial subjective human judgment, because often the true criterion to be optimized cannot be measured, so the humans have to pick the best proxy or proxies possible — and keep these decisions in mind as sources of risk when the models are deployed.
        
        We also need to keep in mind that the data to which we will apply data science techniques are the product of some process that involved human decisions.
        
        The meaning of data is colored by our own beliefs.
        
        Example:
        
        1 more item...
        
        Conclusion from the Example:
        
        2 more items...
- - - - Should we be granted credit?
        
        If so, what should be our credit line?
        
        Should we be targeted with an offer?
        
        What content would we like to be shown on the website?
        
        What products should be recommended to us?
        
        Are we likely to defect to a competitor? Is there fraud on our account?
      - Privacy vs. Improving Business Decisions
        
        There seems to be a direct relationship between increased use of personal data and increased effectiveness of the associated business decisions.
    - - We bring this up to emphasize that privacy concerns are not some easy-to-understand or easy-to-deal-with issues that can be quickly dispatched, or even written about well as a section or chapter of a data science book.
        
        If you are either a data scientist or a business stakeholder in data science projects, you should care about privacy concerns, and you will need to invest serious time in thinking carefully about them.
- - - - Web systems like Amazon, oDesk, etc, are a vast of independent contractors via internet.
        
        Micro-outsourcing is particularly relevant to data science, because it changes the economics, as well as the practicalities, of investing in data.[
        
        Example: (Applying Supervised Modeling)
        
        We need to have specified a target variable precisely, and we need to actually have values for the target variable for a set of training data.
        
        In certain cases, we can use micro-outsourcing systems such as Mechanical Turk to label data
        
        Example: Advertisers would like to keep their advertisements off of objectionable web pages, like those that contain hate speech.
        - Problem: It would be far too costly to have employees look at them all. We might immediately recognize this as a possible candidate for text classification we can get the text of the page, represent it as feature vectors as we have discussed, and build a hate-speech classifier. Unfortunately, we have no representative sample of hate speech pages to use as training data.
        
        Solution:
        
        2 more items...