1. Solution Evaluation
  • To assess the performance of and value delivered by a solution in use by enterprise
  • To rcm removal of barriers or constraints that prevent the full realization of the value

Stages of development:

  • Prototypes or Proofs of Concept
  • Pilot or Beta releases
  • Operational releases
  • Analyze the actual value being delivered
  • Identify limitations which may be preventing value from being realized
  • Make recommendations to increase the value of the solution

May combine:

  • performance assessments, tests, and experiments
  • objective and subjective assessments of value
    => generally focus on a component of an enterprise rather than the entire enterprise

Tasks:

  • Measure solution performance: determine the most appropriate way to assess the solution's performance
  • Analyze performance measures: understand the value solution delivers to the enterprise and SH & determine if it's meeting current business needs
  • Assess solution limitations: investigates issues within the scope of a solution
  • Assess enterprise limitations: investigates issues outside the scope of a solution
  • Rcm actions to increase solution value: identify and define actions the enterprise can take to increase the value delivered by a solution

Inputs

Implemented solution (external)

6.1. Current state description

6.2. Business Objectives

6.2. Potential value

Outputs

Solution Performance measures

Solution Performance analysis

Soluiton limitation

Enterprise limitation

Recommend actions

8.1. Measure Solution Performance

.1. Purpose: to define performance measures and use the data collected to evaluate the effectiveness of a solution in relation to the value it brings

.2. Description

Determine the value of a newly deployed/existing solution

The measure used depend on the solution, context and how the org defines value

Performance may be assess through:

  • KPIs aligned with enterprise measures
  • G&O for a project
  • process performance targets
  • tests for a software application

.3. Inputs

  • Business Objectives
  • Implemented solution (external)

.4. Elements

.4.1. Define Solution Performance Measures

BA determine if current measures exist, or if methods for capturing them are in place

BA ensure that performance measures are accurate, relevant and elicit any additional measures identified by SHs

Common sources of measures: business goals, objectives and business processes

May be influenced/imposed by 3rd parties: solution vendors, government body or other regulatory organizations

Types and nature of the measurements are considered when choosing the elicitation method

May be:

  • Quantitative
  • Qualitative

4.2. Validate Performance Measures

Ensure that the assessment of solution performance is useful

Specific performance measures should align with any higher-level measures that exist within the context affecting the solution

Sponsor or any SH with decision making authority can decide measures used to evaluate soluiton performance

4.3. Collect Performance Measures

BA consider:

  • Volume or Sample Size
  • Frequency and Timing
  • Currency: recently > older data

8.2. Analyze Performance Measure

.1. Purpose: to provide insights into the performance of a solution in relation to the value it brings

.2. Description

The measures collected in 8.1 often require interpretation (understanding) and synthesis to derive meaning and to be actionable

To meaningfully analyze performance measures:

  • BA require a thorough understanding of the potential value that SH hope to achieve with the solution
  • Consider variables: G&O, KPIs, the risk tolerance of both SHs and the enterprise, other stated targets

.3. Inputs

Potential Value

Solution Performance Measure

.4. Elements

.4.1. Solution Performance vs Desired Value

A solution might be high performing, but contributes lower value than expected (or than contribution in the past) OR low performing but potentially valuable
=> BA examines the collected measures to assess their ability to help SH understand the solution's value

If the measures are not sufficient to help SHs determine solution value, BA:

  • collect more measurements or
  • treat the lack of measures as a solution risk

.4.2. Risk

.4.3. Trends

BA consider the time period when the data was collected to guard against anomalies (~abnormal) and skewed trends when analyzing performance data

Any pronounced and repeated trends are noted

.4.4. Accuracy

BA test and analyze data collected by the performance measures to ensure their accuracy

To be considered accurate and reliable, the results of performance measures should be reproducible and repeatable

.4.5. Performance Variances

Performance Variances = the difference btw expected and actual performance

Root cause analysis may be necessary to determine the underlying causes of significant variances within a solution

.8.3. Assess Solution Limitations

.3.1. Purpose: to determine the factors internal to the solution that restrict the full realization of value

.3.2. Description

identify the root causes for under-performing and ineffective solutions and solution components

may be performed at any point during the solution life cycle: development/full implementation/existing solution => regardless of the timing, the assessment activities are similar and involve the same considerations

.3. Inputs

  • Implemented solution (external): a solution that exists
  • Solution performance analysis

.4. Elements

.4.1. Identify internal solution component dependencies

  • Solutions often have internal dependencies that limit the performance of the entire solution to the performance of the least effective component
  • BA identify solution components which have dependencies on other solution components, then determine if there is anything abt those dependencies or other components that limit solution performance and value realization

.4.2. Investigate Solution Problems

  • BA identify problems in a solution/solution component by examining instances where the outputs from the solution are below an acceptable level of quality or where the potential vlaue is not being realized.
  • Problems may be indicated by an inability to meet a stated goal, objective or RQM, or a failure to realize a benefit that was projected furing the tasks Define Change Strategy or Rcm Actions to Increase Solution Value

.4.3. Impact Assessment

  • BA review identified problems -> assess the effect on the operation of the org or the ability of the solution to deliver its potential value => requires:
  1. determining the severity of the problem
  2. the probability of the re-occurence of the problem
  3. the impact on the business operations
  4. the capacity of the business to absorb the impact
  • BA determine which problems must be resolved, which can be mitigated (make sth less serious) through other activities or approaches, which can be accepted

Other activities or approaches may include additional quality control measures, new or adjusted business processes or additional support for exceptions to the desired outcome

BA assess risks specifically to the solution and potential limitations of the solution.

.8. Outcome

Solution limitation

8.4. Assess Enterprise Limitations

.1. Purpose: to determine how factors external to the solution are restricting value realization

.2. Description

Solution may operate across various org within an enterprise => have many interactions and interdependencies

Solutions may also depend on environmental factors that are external to the enterprise

Enterprise limitations: culture, operations, technical components, SH interests or reporting structures

May be performed at any point during the solution life cycle

.3. Inputs

  • Current state description
  • Implemented (or constructed) solution (external)
  • Solution performance analysis

.4. Elements

.4.1. Enterprise Culture Assessment

  • Evaluates the extent to which the culture can accept a solution
  • If cultural adjustments are needed to support the solution, the assessment is used to judge the enterprise's ability and willingness to adapt to these cultural changes
  • BA perform cultural assessment + evaluate internal and external SH

.4.2. SH Impact Analysis

  • Provide insight into how the solution affects a particular SH group

BA consider:

  • Functions: the process in which the SH uses the solution: inputs SH provides into the process, how the SH uses the solution to execute the process, what outputs the SH receives from the process.
  • Locations: the geographic locations of the SH interacting with the solution
  • Concerns: the issues, risks, and overall concerns the SH have with the solution

4.3. Org structure changes

  • The use of a solution and the ability to adopt a change can be enabled or blocked by formal and informal relationships among SHs
  • The reporting structure may be too complex/simple to allow a solution to perform effectively
  • Assessing if the org hierarchy supports the solution is a key activity
  • BA consider informal relationships + formal structure

4.4. Opertional Assessment

  • To determine if an enterprise is able to adapt to or effectively use a solution
  • Identify which processes and tools within the enterprise are adequately equipped to benefit from the solution, and if sufficient and appropriate assest are in place to support it

BA consider: p180

.8. Outputs

Enterprise limitation

8.5. Recommend Actions to Increase Solution Value

.1. Purpose

To understand the factors that create differences btw potential value and actual value, and to rcm a course of action to align them

.2. Description

Focus on:

  • Understanding the aggregate of the performed assessment
  • Identifying alternatives and actions to improve solution performance and increase value realization

Identify how a solution should be replaced, retired or enhanced

Consider long-term effects and contributions of the solution to SHs

Include rcmdations to adjust the organization to allow for maximum solution performance and value realization

.3. Inputs

  • Enterprise Limitation
  • Solution Limitation

.4. Elements

.4.1. Adjust Solution Performance Measures

When the performance of solution is acceptable but not support to fulfill business O&G => require more appropriate measures

.4.2. Recommendations

Do nothing:

  • When the value of a change is low relative to the effort required to make the change
  • When the risks of change significantly outweigh the risks of remaining in current state

Org change:

is a process of managing attitudes about, perceptions fo and participation in the change related to the solution

refer to a process and set of tools for managing change at an org level

BA may help to develop rcmdations for changes to the org strucutre or personnel

Possible rcmdations:

  • automating/simplifying the work people perform
  • improving access to info

Reduce Complexity of Interfaces => imporve understanding

Eliminate Redundancy: different SH groups may have common needs that can be met with a single soluiton => reducing cost of implementation

Avoid Waste: to completely remove activities that not add value and minimize activities that not contribute to the final product directly

Identify Additional Capabilities:

  • Solution options may offer capabilities to the org above and beyond those identified in the RQMs
  • These capabilities are not of immediate value to the org but have the potential to provide future value

Retire the Solution

consider the replacement of a solution/solution component

Additional factors that may impact the decision of replacement/retirement of a solution:

  • Ongoing cost vs initial investment
  • Opportunity cost
  • Necessity
  • Sunk cost

.8. Outputs: Recommended Actions