Please enable JavaScript.

Coggle requires JavaScript to display documents.

Tips for the Class (2 Tracks (2 Tracks (Research Project: Investigate a…

- - - - In a research project, you're typically trying to investigate a phenomenon that already exists. The goal is to gain greater understanding of the nature of the phenomenon and why it exists. Research is often thought of more in the context of the natural sciences, but research in EdTech is a massive field as well. Some general topics for research would include how certain tools actually change the learning process, how schools with technology differ from schools without, how different audiences receive and view educational technology differently, etc.
        If you choose to do a research project, you'll want to:
        
        Understand and very clearly define the phenomenon you want to investigate. Sometimes defining the phenomenon may even be part of the research.
        
        Understand the current theories and methodologies that have been applied to this phenomenon.
        
        Understand the research methodologies that would give the data necessary to make useful statements about the phenomenon.
        
        If you choose to do a research project, your ultimate deliverable will be the results of your research, including the data you gathered, the analysis you performed on that data, and any accompanying documentation describing the structure of that data. In the limited amount of time we have, you may not be able to do a comprehensive study, but you should ideally be able to cycle through the research process (from research questions to data collection to analysis to conclusions) once. Note also that research has some additional IRB and FERPA rules that may need to be followed, so if you're considering the research path, discuss it with your mentor early so we can get the documentation going as fast as possible and maximize the time to actually do your research.
    - - In a design project, you're typically trying to solve some problem by designing a tool to address it. Many of the tools you use in the OMS program are solutions to problems. Piazza and other online class forums address the problem of hosting class discussions without a co-located class. HipChat and Slack address the problem of making that communication more synchronous, rapid, and conversational. OSCAR addresses the problem of getting students registered for classes without gathering everyone in a giant gymnasium the way we used to do.
        If you choose to do a design project, you'll want to:
        
        Understand the problem, including the different stakeholders.
        
        Understand the current systems that try to solve parts of that problem, especially their strengths and weaknesses.
        
        Understand the broader pedagogical, cultural, or organizational issues at play in this problem.
        
        If you choose to do a design project, your ultimate deliverable will be a tool you create to address the problem. The scope and completeness of that tool will depend on the problem you choose to address; very big problems may only be able to addressed with comprehensive prototypes, while smaller problems may lead to actual tools.
    - - The above two categories aren't mutually exclusive, however. There are lots of places where some overlap can occur. Oftentimes, we might use the results of some research to reflect on the design of a tool, or conversely, we might use a tool to test out some research hypotheses. Consider the following:
        
        A quantitative study on how a particular tool exacerbates gender bias in online classrooms, culminating in changes to the tool to counter those tendencies.
        
        A tool to improve student engagement in a classroom, followed by a study to examine whether or not student engagement actually improved through using the tool.
        
        A tool to improve student learning based on a particular pedagogical theory, followed by a study to examine whether or not student learning improved, which then informs analysis of whether or not the pedagogical theory was accurate in the first place.
        
        A qualitative study on the design strengths and weaknesses of an existing tool, like Piazza, that then informs the design of a script that adds valuable extra features or addresses certain problems.
- - - - Read the introduction
        
        If you're still interested in the paper after reading the abstract, read the introduction. Look at the structure of their argument. Map the paper's topic in your mind to other projects with which you're familiar.
      - Read the conclusion
        
        Find out what the paper is claiming. You likely already had an idea of this from the abstract, but read the conclusion to get it in more detail.
      - Read the abstract
        
        This will give you a good, high-level outline. Generally, this will tell you if you're actually interested in reading the rest of the paper.
      - Read other relevant sections.
        
        If you still have questions after reading the conclusion, then return to the body of the paper and read further. In my experience, oftentimes you'll be satisfied simply hearing the conclusion of the paper, and you'll be ready to move on. For papers that are more narrowly applicable to your work, though, you may be interested in learning more about the design of their tool or their methodology because you may want do something similar in your own project.
    - - "How to Read a Research Paper" , from Spencer Rugaber, Professor of Computer Science at Georgia Tech, and the original instructor for our own Software Architecture & Design Course
      - "How to Read a Paper" , from Srinivasan Keshav, Professor of Computer Science at the University of Waterloo
      - "How to Read a Technical Paper" , from Jason Eisner, Professor of Computer Science at Johns Hopkins University
      - "Efficient Reading of Papers in Science and Technology" from Michael J. Hanson and Dylan J. McNamee
      - "How to read a research paper" , from Prof. Michael Mitzenmacher, Professor of Computer Science at Harvard University
- - - - Oftentimes when you're searching on Google Scholar, you'll come across many articles which you cannot directly access. Sometimes the abstract gives you all you need to know, but other times you want to see the rest of the article. In these cases, the Georgia Tech Library will often have a subscription to that journal, allowing you free access to the article so long as you're logged in through the library. See the library web site for more information.
        Remember, though, that when accessing articles that require your Georgia Tech login, you shouldn't share them with any non-Georgia Tech students.
      - Georgia Tech Library
    - - Once you've found one article that's at least close to what you want to find, go to its reference list. You'll typically find dozens of references to other articles. With luck, one or more of these will be closer to what you actually want to find. Like a trail of breadcrumbs, you can follow these citation lists through the history of a community.
        Some article hosts will actually parse and show the references as hyperlinks to those articles elsewhere on the host or on the internet. Other times, you'll need to return to Google Scholar and search for the title.
    - - If there's some topic about which you're interested in learning more but you just can't seem to find some information, don't hesitate to ask! Ask on the class forum especially: your classmates may have found some sources in which you'd be interested, and they might be interested in what you find as well. That may also help you find other people with similar interests for future collaborations.
    - - If you're not familiar with Google Scholar already, you're going to fall in love with it very quickly. Google Scholar is Google's search engine for academic publications (as well as patents and some other formal information). It gives you the ability to filter by dates, and generally has a pretty good keyword search. It may not find you exactly the article you want, but it will usually find you one that will help you get started with the next couple tips.
        Note also that you can use Google Scholar to search for specific authors as well using the author search tool. For example, here are the search results for my name, and here is my author page on Google Scholar.
      - Google Scholar
      - Author Search Tool
      - Example Author Search
      - Example Author Page
    - - Similarly, just as you can use an article's reference list to find similar articles that might be closer to what you want, you can also simply trace back through the author's own publication history. Search for the author's name, and you'll generally find either their personal web site with their CV and articles or their profile on some aggregator, like Google Scholar or ResearchGate. Then, you can go back through their own articles.
        For example, you might at some point come across an article that builds on a claim the author has made in the past. That claim made in the past might actually be what you want to read about, so look back through that author's publication history to find when they may have originally made that claim.
- - - - If you choose to share your work outside the class, there are a number of opportunities you may be able to leverage:
        
        Talk to experts in the field in which you're interested! You're free to reach out to them directly to see if anyone would be willing to chat about your ideas. The teaching team and your classmates also may have valuable contacts that you can use, so if you're interested in talking to someone in a particular area, post as such in the class forum and see if anyone can connect you. If you want to speak to anyone at Udacity, I'm certain I can set that up, and I'm sure your classmates and the members of the teaching team will have contacts in local schools, in the EdTech industry, or in academia.
        
        Post your work to your personal web site, blog, or social media site. Many students already do this at the end of the semester, but in this class where you're encouraged to solicit feedback and ideas, why wait? Post it during the semester and see if you can get feedback from the outside world.
        
        Post your work to the rest of the class, or with the rest of the OMS program. You'll already be getting feedback from a few classmates and your mentor during peer review, but if you think there might be certain classmates with backgrounds better-suited to your work, go ahead and share your work with them, too. This may be especially valuable if you're looking to design a tool to use here in the OMS program â€” you have a readily accessible body of potential users to query via Google+, the Facebook page, HipChat, or Slack.
    - - We will never share your work without your consent. In the class surveys, we will ask for your consent to add your project presentation and paper to this course library, which would share it with future students in this class, and with the public. If you would prefer, we can also exclusively share your work with future students in the class without making it publicly available. You are also welcome to share it on your own personal web site, and we will add it to the library as a link to your site.
- - - - Things to Research
        
        Student Success:
        
        What predicts student success? Is student success most predicted by factors of the students as they come into the program, or are there things that we or they can do that will improve their likelihood of success?
        
        Plagiarism:
        
        What are the motivations behind plagiarism? Are they typically students who are overwhelmed and resort to plagiarism as a final option, or are they typically students that look to plagiarism as a first plan? Can we prevent it structurally (by providing the resources necessary to succeed) rather than preventing it by catching and penalizing it?
        
        Peer Feedback:
        
        How does peer-to-peer feedback improve the classroom experience for participants? How does access to peer review improve the feedback students get from graders? Does access to peer review change the grades that graders ultimately give to students?
        
        Retention:
        
        What factors predict whether students will remain in the program? Are they characteristics of students as they enter the program? Is it something that happens while they're in the program? Can we improve retention, or is it strictly a function of the students themselves?
        
        The Harvard Business School (HBS) Case Method:
        
        The Harvard Business School Case Method is a way of approaching business courses to engage students. Though it was developed at HBS other institutions leverage the methodology for teaching their students. Because much of the method revolves around synchronous discussions in the same space some faculty do not believe it is equitable to have an online method. However, with many schools moving degrees online is the belief that a video conference is equitable true? If so does there have to be certain settings in the setup such as the instructor being able to see all participants while they are not talking to see their engagement level? Can you achieve the same outcomes with an asynchronous setup, and what would it look like? If nothing exists currently can something be built to achieve the same level of outcomes?
        
        Gender
        in the OMS:
        
        diSalvo's and Guzdial's research notes a number of obstacles to women enrolling in the OMS: perceived animosity toward women in an online environment, perceived questions on the quality of the program, and other issues, leading to a higher gender disparity than on-campus programs. What differentiates women who apply to and/or enroll in the program from those that do not?
        
        Breaking the Isolation:
        
        What experiences, especially based on interactions with teachers and other students, do online students tend to miss out on? What impact does that isolation have on the ultimate learning? What can we do to link them with others to break this isolation a bit?
        
        Connectedness:
        
        Can we quantify how interconnected an online class is based on observable interactions? Can we connect those models of interconnectedness to levels of classroom success or failure? Can we make changes that encourage those types of interconnectedness that connect to classroom success?
        
        Meta-Success:
        
        How do we define a successful class? Success at the student level is a common idea, but how do we define a class as a whole as successful? What metrics do we use? How do we populate those metrics? How do we validate those metrics?
        
        Angels and Devils:
        
        Some classes have been disproportionately impacted, positively or negatively, by the presence of certain individual students. Can this effect be quantified and replicated or prevented?
        
        The Halo Effect:
        
        Why do students react so differently to glitches and errors in some courses than others? Sometimes students are very understanding and patient when things go wrong, and other times they are incredibly critical. What determines which occurs? Can we create positive environments that encourage the former rather than the latter?
    - - Things to Build
        
        Group Formation:
        
        Many classes rely on group projects, but forming students into groups for group projects is an arduous process. Different classes have different constraints, and different students have different priorities. Develop a tool that could gather information about students and make recommendations about groups based on criteria and preferences entered by students and instructors.
        
        Detecting Plagiarism:
        
        Tools like TurnItIn already help detect plagiarism in written work, but with the requirement that work be then included in an archive for future users to test against. Can plagiarism instead be detected at the local level based solely on a canon of past work submitted in a class? There exist tools that do similar analysis as TurnItIn for programming, but they are not yet perfectly suited specifically for plagiarism detection. Additionally, code presents many more fair opportunities for copying that does not constitute plagiarism. Develop a system that addresses one of these problems, either detecting plagiarism of written work only within a local database or detecting code plagiarism with the ability to ignore cited code.
        
        Intelligent Tutoring Systems:
        
        Intelligent tutoring systems have been effective in a number of domains, like Algebra and Geometry. Build one for a new domain!
        
        Managing Human Grading:
        
        Even as we continue to rely on humans to grade written or open-ended work, there are still a lot of places where technology can help. MOOCs rely on peer-to-peer grading, but there are questions about its reliability. Our OMSCS classes use expert human graders, but consistency between graders is not a given. Construct a system that will manage the grading process and automatically identify and emphasize more reliable peer-to-peer graders or identify and resolve differences between expert graders.
        
        Academic Adviser:
        
        Develop an AI agent that can serve as a virtual academic adviser for students, answering questions on things like schedule formation, class prerequisites, readiness questions, etc. Ideally, construct it to take into consideration the student's own personal information, recommendations and reviews from past students, and intelligence regarding what kinds of students have succeeded and failed in other classes to allow recommendations like, "You might want to take Introduction to Operating Systems before taking Advanced Operating Systems".
        
        Essay Autograding:
        
        Can essays be autograded? We'll likely always want to have human experts sign off on the quality of essays, but human grading takes a while and leads to a slower feedback cycle. Develop a system that can autograde essays based on a collection of past good and bad essays. In constructing this system, think especially on how such an autograder could generate immediate feedback for the writer so that they could improve the essay before submitting it. You might also extend this idea beyond just education: imagine an auto-editor for blog posts, news articles, or scholarly publications.
        
        Remedial Classes:
        
        This semester, Chris Pryby of Udacity has developed a new linear algebra refresher course to help students get prepared for classes like Computer Vision and Computational Photography. There are other subjects that students have reflected needing as well, like C and MATLAB. Develop a remedial course for one of these. Or, similarly, there are likely topics in other domains and other universities on which students need to be refreshed before moving forward. Develop a remedial class for any other topic. Or, for that matter, one person's remedial class is another person's regular class â€” so, develop a class!
        
        Attendance:
        
        For financial aid and some other programs, students need to demonstrate a certain attendance record to qualify for their aid. Attendance has no clear meaning in an asynchronous online course, however. Develop a definition for attendance in this context informed by the rationale behind these requirements, then develop a system that can actually monitor for attendance according to that definition.
        
        EdTech Library:
        
        Could this library work better in a different format? Perhaps something constructed on WordPress that would allow more extensive linking and embedding? Maybe something that would lend itself to targeted discussion on specific topics that could then be aggregated into a more traditional forum? Reinvent this library in a way that makes it easier to find and navigate among topics and easier to have discussions. Remember that in doing so, you're actually creating a solution that could be used for any library like this.
        
        Office Hours:
        
        Right now, we use at least three different tools for office hours in the OMSCS program: Webex, Google Hangouts, and Google Hangouts on Air. Each has its own benefits: the static URL and waiting room function of Webex, the easy recording and rebroadcasting of Hangouts on Air, the simple setup of Hangouts. Additionally, all three are missing some features we may need, like the ability to easily divide and recombine rooms for targeted discussion. Construct a single office hours solution that handles all the office hours uses cases we run into: 1:1 conversations, 1:many conversations, recording and rebroadcasting while remaining FERPA compliant, etc.
        
        Class Projects:
        
        You've taken a couple OMS courses already. Could the projects in those courses have been improved? Do you think you could modify the project descriptions to better bring out the learning objectives, or develop autograders or other tools to give more immediate feedback on progress on those projects? Could you create a tool to allow assignments for CS6505 to be completed on screen, or an evaluator for getting instant feedback on the performance on the Test and Raven's problems for a project in CS7637? Take a crack at redesigning those past projects.
        
        Forums:
        
        We've seen fantastic forum discussions in the OMSCS program, but at the same time, many students have noted several things about our current forum solution that aren't ideally suited for this program and these classes. Build a new forum system that's custom-tailored to the kind of closed, large, instructor-led classes we have in the OMS program. Some things to consider include: facilitating both Q&A and discussions; facilitating group work; managing information overload; facilitating both cursory browsing and in-depth search; managing official communications; and integrating with other tools.
        
        TA Symbiosis:
        
        A lot of people in this class are TAing other classes, and ideally, many of you will go on to become TAs. A side goal for this class is that it would become a training ground for future TAs across the program to learn about education and technology. Use that alignment to improve the other classes on which you're working. Your EdTech project can be improving a tool specifically for another class in which you're involved, giving you a live test audience for whatever you develop.
        
        Identity Verification:
        
        The one edge case that has not yet been addressed (that we know of) in education (online or in-person) is the possibility of someone completing work on behalf of a student. Because the work is original, it would not appear in any online database, but yet still would not be the student's own work. Proctoring addresses this for short-term assignments, but we clearly do not want to proctor a student writing a 10-page essay in a single sitting or something. Develop a solution that would allow a school to proctor a shorter essay, then use machine learning to determine if future essays were written by the same student that wrote the proctored essay.
- - - - It's fortunate that you're interested in both adaptive tutoring and game-based learning, as they do have the potential to nicely feed into each other! Games are a great platform to deliver intelligently-tailored content in an organic, somewhat-opaque way. Do you think you'd be interesting in creating a small game as a platform to test out your proposed system? Or are you perhaps leaning towards working on a more general solution?
        In any case, connecting ML to education seems like it'd open up a whole slew of interesting potential design projects, and it's going to be a great place to start! Enjoy the term!
      - That's really fascinating. I like the idea of basically having a system that heavily uses meta-cognition. Although it is definitely not the normal use of meta-cognition. When I normally think of that, I usually think of a situation where an agent learns from itself but in this situation, it sounds like it learns from feedback, which is still fascinating idea. One of the things I remember most about KBAI is how we were able to see links between humans and AI and how studying one helps us understand the other. This seems like a great way to explore those ideas further than you would otherwise. I am excited to see where this goes.
      - Language education is an area that has seemed to really embrace technology. As speech recognition software has improved it has really aided in the delivery of spoken language education, now appearing even in the mobile space. It would be interesting to see how some of those systems can compare to receiving education in a more traditional setting. Having taken a few language classes and always struggled, one of my big problems was trying to speak in class due to a sense of embarrassment. I worried so much that I was saying something wrong that I never practiced as much as I should. Computer based systems could do a lot to help students like myself.
    - - Group work is an interesting topic. I'm curious how effective it is vs solo work for learning. I have known teachers to cite it as less work for them, but students often complain about their teammates.
        I would add to your list cooperation as a question. Communication, coordination, and evaluation rely on cooperation as a basis. The work I am doing this semester is on examining cooperation in group work from the perspective of game theory. I'm curious if you could build mechanisms to aid teachers in evaluating the cooperation of group members?
        What do you imagine the end result of your research will be? A summary of current publications? A survey on what students and teachers like and use? Or will these be purely facilitative toward creating an app, whose structure is unclear at this time because you still need to have research results?
        How feature rich to you imagine this app would be this semester? You might not know which features to select, but do you have an idea of how many features you can support? Have you thought about platform basics? Integrations?
      - "Highly impressed by the research you have done thus far!
        As I was reading your paper I reflected on my two bachelor degrees and how different the advising was. One was 1-1 with an assigned professor, who would ask me what courses I would take next, give suggestions, ask how things were going, ask for feedback on our department's courses, etc. The other was done in email and unless something was going wrong, they didn't say much.
        My own experience with the OMS CS program is similar to what you recorded. It is primarily about determining what to take next. Also as you say, we have lots of options, in fact I dropped both classes I originally signed up for this semester in order to take this course and the Machine Learning for Trading course. It felt very strange doing that, but we have the flexibility to do so.
        Overall, your idea of building some sort of automation or path or guide to helping other OMS CS students in their choice of classes is not only admirable, but do able. There are so many questions and opinions that keep coming up over and over and I think capturing these would help so many people (and deal with the scaling problem)."
      - Good overview of some of the problems faced when attempting to train competent programmers. You identify some clear gaps in traditional teaching methods, and make some good proposals on how these gaps can be overcome. Project based learning, in particular, has been of great value to me in in terms of providing sufficient motivation to delve deep into various technologies. There have also been some good games that do exactly what you propose. https://microcorruption.com/ is one of my favorite examples of this. I encourage you to keep exploring this area and hone in on the very specific problem you hope to solve this semester.
    - - I think your subject reaches a general point of weakness. I had one 'shop' type class in my whole time in school. I would have loved to have a hands on type class where I could have done this kind of thing or maybe some other fun stuff. The only class I had was a class where I think the most advanced thing I did was create a car that was a moved by an air cartridge. Having said that, it was only for like half a semester and I never touched another thing like it since then. Having worked in Education, most of the 'shop' type classes seemed to be associated with lower educational students who needed some skills in life because they were not cut out for normal education.
        With that, I think if we could have more robot type things involved I think it would be awesome for a lot of people because I am confident that MANY people could do some very basic programming logic, even if its only using the block type programming. So I guess what I am saying is to some degree improving the level of robotics might come down to getting more robotics in schools such that the tools can be more required and then hopefully cheaper.
      - Cool project focus! I think a small percentage of the class projects will actually focus on generating course content. Because you're so well focused and succinct here I actually don't have too much to say.
        It sounds like you have a good understanding of what is and isn't effective in these types of courses. Based on my experience I'd agree that interactive courses tend to be the most effective. But, there's an art to producing constructionist content like this. You need to strike a balance between easy-mode online tools and setting up an intricate development environment on a target computer. Perhaps the latter's a little easier because you can focus on OS X, but it should be a consideration nonetheless.
        That, in my opinion, is the crux of success for courses like the one you're proposing. It may even be worth investigating that specific topic more. What, in your experience, is the right level of DIY versus online-editor-type-stuff in courses like this? In every aspect, carefully consider who you're targeting and what type of delivery is most suitable for them.
        Overall, it sounds like a great project and I hope you share when you're finished!
      - I love the idea of bridging the gap in education for kids with lower income --- I am not sure if there are already programs tackling that since I do not have a teaching background. There are a couple suggestions I could make:
        
        organize the reflection around sections for problem definition and potential solutions.
        
        add references of utilized literature (so far you have only added one)
        
        describe in a little more detail how Scratch and similar tools can solve the problem you are trying to address. You did a pretty good job describing the the potential effect of introducing them to low income communities. It would be interesting to see in the proposal what exactly are you going to do about them and how far are you going to go?
        
        What do you envision the outcome of the study to be?