Please enable JavaScript.

Coggle requires JavaScript to display documents.

Chapter 13. Startup Processes (Chapter 14. Feature Understanding and…

- - - - If the data is stored in a database such as PostgreSQL, Oracle, or MySQL, select ODBC (Open Database Connectivity--a standard for connecting to just about any database)
        
        If the data is in a Hadoop system and you have an Enterprise DataRobot account, select HDFS.
        
        If on a slow internet connection, compressing the data as one of the following file types can speed up data upload: .gzip, bzip2 or zip archive.
      - Select Local File (Local File) -- To select the appropriate file, navigate to the directory where data for this book is stored and open the A.1 Hospital Readmissions directory and select the file named 10kDiabetes.csv and click open.
        
        While the data is being processed click on Untitled Project in the upper-right part if the screen and name the project
        
        Create tags to store your data and make it more easily accessible in the future
        
        Be aware that any project can be shared with collaborators by clicking the share icon, DataRobot documentation can be accessed with the documentation icon, customer support with the megaphone icon, and account and settings can be managed with the user icon.
    - - Excel files (.xls and .xlsx) may be the safest bet, particularly if the data is small enough to be read into Excel (about one million rows or less).
- - - - The index number is used to specify what features are being discussed.
        
        Unique: notes how many unique values exist for each specific feature. For example, the rowID column has unique numbers from 1-10,000 so there are 10,000 unique values. The next feature, race, has only 5 unique values.
        
        DataRobot offers to expand feature details if the feature name is clicked. The feature list will expand to show details about the race feature. Any feature name can be clicked to display more details about the Feature. When a new feature is selected, DataRobot will close the previous feature detail screen.
      - To the right of the Unique column lies information on standard descriptive statistics such as Mean, Std Dev(standard deviation), Median, Min, and Max. These stats are only available for numeric columns (categorical values, mean would be replaced with mode—the most common value—which can be displayed visually by clicking into the details screen).
        
        When using histograms in future presentations, be ready for the question about where the cutoffs exist between bars
        
        The mean is listed as 4.43, suggesting that the average hospital stay is a bit more than four days with a standard deviation of 3.02, which indicates that, if this had been a normal distribution, the 68-95-99.7 rule would apply. This means that 68% of the data is within the -1 to 1 standard deviation, 95% of the data is within -2 to 2 and so on
        
        To do this, select the orange down arrow in (showing 7 bins), and change it to 14 bins. Figure 14.6 shows that, for the first bar, the original 7 bins “bin range” is between [1,3), while the range of the 14 bins is [1,2). The difference between brackets [] and parentheses () is important. A bracket denotes an inclusive range whereas a parenthesis denotes an exclusive range. In other words, the range includes the number next to a bracket, but not the number next to a parenthesis. Here are all possible values for the first bin in the 7-bin range:
        (1, 3) = 2
        (1, 3] = 2,
        [1, 3) = 1,
        [1, 3] = 1, 2, 3
        
        Robot will only present the data in the third position above, bracket followed by parenthesis, meaning that the first number is counted in this bar on the histogram, but the last number is not. In Figure 14.6, the left figure shows that for the 7-bin case, the first bar contains all ones and twos. In the right part of Figure 14.6, in the 14-bin case, the [1,2) indicates that it contains a count of only the ones, as one would expect. So, there are 1,410 values of ‘one’ in this feature. An easy way to remember what the numbers mean is to mentally place a less-than sign in front of each number. When it states a 3, turn it into a <3 (lessthan 3, not a heart), but remember that it only indicates the next value less than that number, such that <5, or [3,5), contains the numbers 3 and 4
        
        Data Types:
        
        A data type indicates the nature of the data inside of a feature.
        
        Variable only has two categories (binary categorical data), It is also common to have many categories (multi-class categorical)
        
        The other most common data type is numeric, which can be any type of number, including integers and decimals.
        
        1 more item...
        
        Evaluations of Feature Content
        
        Take note that the feature rowID at index1 in Figure 14.1 has been detected as a unique identifier and marked with [Reference ID]. This means that the system will not use it to predict your target.
        
        1 more item...
        
        Missing Values
        
        1 more item...
  - - - Advanced Options: Examine the advanced options. Click “Show Advanced Options” (), For now, ignore the details below Smart Downsampling.
        
        Starting the Analytical Process: The analytical process, which will prepare the data through the prescribed options: Autopilot,Quick,and Manual
        
        Quick Run is an abbreviated version of Autopilotthat produces almost as good models (on average) by shortcutting the DataRobot best practice machine learning process. Autopilot and Quick are identical except that for Autopilot DataRobot starts the analysis at 16% of the sample and using that information to determine which models to run with 32% of the sample. Quick starts right at 32% with models that have historically performed well. e other difference is that in Quick, only four models are automatically cross-validated and only one blender algorithm applied.
        
        Know that Informative Features represents all the data features except for the ones automatically excluded and tagged, for example, as [Duplicate] or [Too Few Values].
        
        Autopilot will implement the standard DataRobot process and likely lead to the best possible results. This
        
        Step 1, “Setting target feature,” transfers the user’s target feature decision into the analysis systems.
        Step 2, “Creating CV and Holdout partitions,” uses the decisions we made in the advanced settings (or the default if we did nothing) to randomly or semi-randomly assign cases to the holdout and various cross-validation folds.
        Step 3, “Characterizing target variable,” is where DataRobot will save the distribution of the target to the analysis system for later use in decisions about which models to run.
        Step 4, “Loading dataset and preparing data,” is relevant if a) the dataset is large (that is, over 500MB); b) all the initial evaluations before this step will have been conducted with a 500MB sample of the dataset (or all of the dataset is smaller than 500MB); and c) now the rest of the dataset is loaded.
        Step 5, “Saving target and partitioning information,” is where the actual partitions are stored in cross-validation folds, and holdout sets are stored in a separate file on a disk.
        In Step 6, importance scores are calculated (discussed these in the next paragraph). The features have now been sorted by their importance in individually predicting the target.
        Step 7, “Calculating list of models,” is where information from steps 3–6 is used to determine which blueprints to run in the autopilot process.
        
        1 more item...
        
        There are occasions where it makes sense to increase the holdout sample to more than 20%, such as when you have a lot of data, and you want to ensure that the holdout evaluation is as accurate as possible. For this dataset, it is best to stick to 20%. Since there are 10,000 rows (cases) in this dataset, a random set of 2,000 cases will be pulled out ur dataset and stored in a “lock-box.” These 2,000 cases will be unlocked right before making the final decision about whether to place this system into “production” (start predicting cases from the real world). The only difference between Random and Stratified is that the Stratified option works a bit harder to maintain the same distribution of target values inside the holdout as the other samples (approximately the same percentage of True values inside each sample).
        
        First it is a reminder of the distribution between the various samples split into five folds and a holdout. Second, it specifies that the cross-validation scores to come from the system are the average of the scores for the number of models used (including the validation model). This
        
        Moving now to the Partition Feature (), this is a method for determining exactly which cases are used in different folds. Partition Featureis different from the other approaches in that the user must do their own random or semi-random assignment of cases. It is
        
        this option is helpful when analyzing graph (network) data where it was important to make sure that there was no overlap between friendship networks (groups of people directly connected to each other) in the train and validation sets
        
        The Groupapproach () accomplishes much of the same as with the partition feature, but with some key differences: firstly, it allows for the specification of a group membership feature. Secondly, DataRobot makes decisions about where a case is to be partitioned but always keeps each group (those with the same value in the selected feature) together in only one partition.
        
        1 more item...
      - First note that DataRobot offers the option of which metric to optimize the produced models for. In general, DataRobot can be trusted to select a good measure, but sometimes a customer or client has a preference for another measure on which they will evaluate the performance of the resulting models, in which case, consider using their measure. By selecting the orange down-arrow (),the options for additional metrics are displayed. Note that these are just the top options for this dataset and for this specific target, which is a binary classification problem (True/False). For regression problems (predicting a numeric value), other measures will be proposed
        
        LogLoss (Accuracy) simply means that rather than evaluating the model directly on whether it assigns cases (rows) to the correct “label” (False and True), the model is evaluated instead based on probabilities generated by the model and their distance from the correct answer.