2020 priority (Cloud First Core CI efficiency (Setup/deployment time…
Cloud First Core CI efficiency
Setup/deployment time consuming, up limit the test setup time
put spark binary into docker image
Spike: dummy ssh connection to create theatre object
leverage S2 for testing data: Splunk index
Smart to pick which QA suites to run in PR pre-merge.
How to help developer to triage testing results quickly
Action : provide test guidance document and meeting
Action: filter out known issues
Action: automate all the work with one button (bonked build, run job, filter out potential regression issues, related log in one page with failures )
How to give developer a clear tests results to evaluate the PR quality
See how do we manage comment out test cases
grab the jira ticket from test case and judge if we should run it
Flaky tests analysis and category
Test case cleanup
Block time for triage
Action: Ning figure out how do we allocate and manage time
Action: Ning figure out how this work with Infra team
Spike: Test data converter tool for file format
Tools efficiency ownership in the future.
Ning: talk to infra team and see how do we work together in the future
Action: SPL generation tool to help single command and command combinatation test and some feature test -- QA
Action:tool for narrow down Commands combination issue automatically
What's the test strategy?
How to get the information for testing on developer side, unit test, ctest
How do we know the gap between unit test/Ctest and QA functional tests.
Action1: discuss with developer to get list from unit test/Ctest
Action2: update automation tool to get SPL list
test process efficiency
How to transfer smoothly for commands test from QA own to Developer own
Commands test guidance doc
SPL generate tool
commands combination issue analyzation
provide test data which is richful
How to get a customer close environment feedback.
There is no formal process for DFS customer bugs management
Telemetry does not provide enough customer information
How to leverage customer bug to improve product quality
generate report and share within teams