Please enable JavaScript.

Coggle requires JavaScript to display documents.

Thesis structure, Exploration and analysis - Coggle Diagram

- - - - :question: What is the effect of using multiple model sizes (nano, medium, XL)?
        :checkered_flag: Train different object detector model sizes with similar params/hyperparams to compare results
      - :question: What metric of evaluation was used and how does it relate with the main objective?
        :checkered_flag: Describe main objective -> Avoid FN and analyze IoU
      - :question: In which scenarios the selected model usually fail? is there a pattern? How can we mitigate it?
        :checkered_flag: Analyze results based on terrain, occlusions and shadows
      - :check: Split data equally based on the different conditions (terrain, occlusions,..)
    - - :question: In which scenarios the selected model usually fail? is there a pattern? How can we mitigate it?
        :checkered_flag: Compute the error for the test set and visually check the worst predictions. Add a specific data augmentation step
      - :question: What is the effect of using different sets of keypoints? What is the min set of keypoints? Why we selected 8 and no other number?
        :checkered_flag: Compute the pose error for different configurations and number of keyponts and select the best
      - :question: Which of the selected representations is more convenient for the problem? How the keypoint location is later extracted from the prediction?
        :checkered_flag: Try the three proposed representations and select best based on qualitative and quantitative evaluation
      - :check: Check the distributions of the train and test set
      - :question: What metric of evaluation was used and how does it relate with the main objective?
        :checkered_flag: Describe the main objective and define a metric directly related with the pose -> In plane rotation and translation along the Z axis
      - :question: What is the effect of using different model sizes?
        :checkered_flag: Compute the pose error for 3 model size with diff num of parameters
    - - Different statistics about the dataset
      - Manual labeling process
      - What is the distribution of the training and test set? Is it the same for the test set? is it fair to evaluate on the test set?
      - Main description of the dataset
  - - - :question: Which scenario is more challenging ? Where are the poorest results happening ? Why ?
        :checkered_flag: Analyze performance of the model based on the different terrains, occlusions and shadows
      - :question: Is the model trained on both RGA and RSTA robust enough to detect RGA and RSTA ? Does it make sense to train on RSTA despite the class imbalance ?
        :checkered_flag: Train on ESA dataset on RGA images to evaluate on RGA and train on mixed to evaluate on mixed (RGA + RSTA)
      - :check: Redefine the evaluation metric
    - - :question: What is the influence of the different variations? is the approach robust to shadow and occlusion?
        :checkered_flag: Evaluate the performance of the model on shadow and no shadow. Then evaluate for each type of occlusion (head, tail, contour)
      - :question: What is the influence of the different terrain types on the performance of the model
        :checkered_flag: Evaluate and compute the error for each terrain
      - :check: Redefine the evaluation metric
      - :question: Is the dataset big enough? Does the model improve when using pretrained weights from JPL dataset?
        :checkered_flag: Train a model from scratch and other using pretrained models and check their performance
    - - Data collection
      - Statistics on the dataset (distributions and plots)
      - Manual correction process
      - Definition of train and validation
      - What is the distribution of the training dataset? Is it the same for the test set? is it fair to evaluate on the test set?
  - - - :question: What are the most common drawbacks of the cross detection ? Which are the common fails and mistakes ? Which is the most challenging dataset ?
        :checkered_flag: Find and investigate the causes analyzing the results based on the terrain types, occlusions or shadows.
      - :question: Is the trained model robust enough to be invariant on a different dataset ? Will the performance remain the same/similar ? Does the JPL model trained on gray/color images perform better on ESA dataset ?
        :checkered_flag: Train on JPL and evaluate the best model on ESA dataset and viceversa
      - :question: Does the ESA trained model perform good despite having the glove attached ?
        :checkered_flag: Train ESA only on RGA images and evaluate on JPL
    - - :question: How it is the performance of the JPL on black and white images
        :checkered_flag: Evaluate the best JPL when training and evaluating on bw images
      - :question: JPL vs ESA. On which dataset the performance is better? Which dataset is more challenging
        :checkered_flag: Evaluate the best JPL model on the test ESA dataset and viceversa