Oracle responses

EOUG Conference&Exhibition MADRID 2000

1. SUPPORT

Support Services showed the highest Gap between Importance and
Satisfaction and was the issue most likely to be raised with an Oracle Senior Executive.
The Good News is that in answering another question, (Q29), nearly 50% of the respondents
acknowledged that Support Services had improved over the last 12 months.

It is clear from the verbatim comments that Support remains a very sore
point for many users, and the following quotes are fairly representative:

„Overhaul your support strategy. I understand the use of relatively
inexperienced front-line technical support personnel. But it takes entirely too long to
get the more experienced analysts involved in assisting customers.”

„Fix Support. Whenever a difficult problem is encountered, the
standard response is to blame the operating system.”

„Obtaining technical expertise from Oracle’s Support services is
frequently impossible. Typically in-house DBA’s are significantly more knowledgeable
than the support personnel which are available, which makes support services less than
worthwhile.”

The IOUC believe that the improvement seen over the last year has been
due to the introduction and extension of Metalink, although it is also clear that a large
percentage of customers do not know what benefits Metalink can bring them.

Please have Ian Thacker identify:

  1. What steps are being taking to continue the improvement in Support Services? e.g.
    Faster response times
    Continuous training
    Continuous uptime of support services
    Globalisation of the service
  2. What are Oracle’s plans to ensure that ALL users find out more about Metalink?
  3. Are there any plans to encourage customers to move up the support levels?

ORACLE’S RESPONSE:

Oracle Support Services is aggressively pursuing an e-business
transformation designed to deliver many important benefits for our customers. Oracle is
enhancing currently available internet-accessible services to provide rapid access to
self-service support solutions:

  • Responding to customer demand for access to descriptions and solutions for their
    licensed products, Oracle has made the most recent 18 months of bug resolutions available
    for customer viewing.

  • Knowledge repositories now include certification matrices, product information,
    technical white papers, patch-fix databases, hints and tips and much more.

  • iTAR (TARs filed through the Internet) can be filed at customer convenience.
  • Oracle will soon implement more uniform processes for: managing problem severity,
    processing escalations and routing problems to analysts with appropriate skill sets.

  • Oracle will continue to enhance search tools to provide faster access to solutions and
    better matching of issues and answers.

Many of these facilities are available now through MetaLink, which is
used by over 120,000 users and has over 2 million hits per day. Other improvements are
being incorporated into Oracle’s Customer Relationship Management (CRM) products which
will form the basis of Oracle Support Services’ next generation customer care and call
center support operations environment. Implementation of this new environment will begin
this summer (2000).

2. THE RELATIONSHIP BETWEEN SUPPORT AND DEVELOPMENT

The results of the survey indicated that Oracle’s communication with
its customers had improved since the last survey (Q1), which tends to make problem
situations more tolerable. This is clearly Good News. The Bad News is that from a customer
viewpoint Oracle Support and Oracle Development do not seem to project a „One Company”
image. From the comments made, it is often difficult to get information from Support when
they indicate Development is involved.

Please have Randy Baker, Ron Wohl, and Gary Bloom co-respond by
articulating what actions will be put into place over this next 6 months to more tightly
link Support and Development so we can;

  1. See faster response and resolution times
  2. Get a one-stop shop answer without calling back many times

ORACLE’S RESPONSE:

Recognizing the importance of a tighter linkage between Oracle Support
Services and Development, the support organization now reports to Gary Bloom, executive
vice president reporting to Larry Ellison. Mr. Bloom began his 13-year Oracle career in a
support management role and understands the vital role that Support plays in our
customers’ success. In addition to Oracle Support, Mr. Bloom oversees a number of
development teams, Oracle University, Alliances and Worldwide Marketing. Mr. Bloom has a
long-established reputation for driving an agile, responsive organization that operates
efficiently across lines of business.

In recent months, Oracle Support and Development teams have jointly
pursued a number of key initiatives that will benefit Oracle customers. These efforts
include:

  • Establishing uniform problem severity definitions and handling procedures (in process)
  • Defining time-based escalation procedures for severity 1 issues (in process)
  • Enabling customer searching and viewing of bug descriptions and solutions via MetaLink
    for Internet-speed access to answers (available)
  • Internal implementation of Oracle eSupport products which will offer customer benefits
    in the form of consistent problem management and escalation processes and easy-to-use
    self-service problem resolution services (in process)

3. QUALITY & RELIABILITY OF PRODUCTS

The third most important issue to be raised with Oracle Executives.
Although there were no questions in the survey specifically about Quality and Reliability,
clearly they are issues of concern. The Good News is that the scores seem to be improving
for Oracle products. Functionality and the ability to integrate products posted better
scores overall.

Bad News: Many users continue to cite far too many bugs on early
versions of Oracle’s products. Most believe that Oracle’s goal is still centered
around time to market and getting the next version or product out versus getting existing
production units to work effectively.

The following were some of the comments made;

„Does Oracle feel that they place more importance on releasing new
functionality than on the production of software, which has been fully tested and provides
fixes for exiting software problems? This is how it seems to us as users of Developer 2000
and Designer 2000, particularly following the recent releases of Designer, and it isn’t
a good impression to have.”

„Bugs in new versions. Incredible Designer 6.0 and Developer 6.0 has
so many bugs”

„Premature roll-out of products; too many bugs, not enough
documentation.”

Please have one Development SEVP that can speak for Oracle answer the
following questions and help us understand;

  1. Oracle’s overall approach to Quality and philosophy in continuously driving down the
    number of bugs found in new products versus shipping to meet market demand.
  2. What clear steps will be taken over this next year to measure and improve product by
    product quality?
  3. Can/Will Oracle provide the user community with periodic Quality reports that indicate
    the progress Oracle is making in driving quality up and bugs down?

ORACLE’S RESPONSE:

Over the past several years, the Oracle user community has enjoyed the
benefit of the continued investments Oracle Corporation has made in improving product
development and testing procedures to enhance quality and reliability. The progress made
is shown by improving scores reported on recent IOUC surveys for product quality,
functionality and integration. These benefits are paying off for Oracle in the marketplace
as well, as Oracle continues to enjoy a better reputation for product quality than many of
the company’s competitors.

Several years ago, Oracle instituted a formalized „Product Release Process” (PRP),
which is designed to improve product quality and ensure product and company readiness for
releases of major products. All relevant organizations within the Company (including
development, product management, product marketing, product lines, support and consulting)
participate in the PRP, to coordinate schedules, assure adequate cross-product testing and
integration, facilitate cross-organizational training, etc. The PRP facilitates awareness
to participating organizations and management of product development schedules,
integration issues, and the status of documentation, testing, education and marketing.
This PRP has resulted in better product quality and improved readiness to sell and support
Oracle products worldwide.

Oracle and users share a common interest in the continued introduction
of upgrades and new and innovative, but high quality, products. Oracle Corporation
recognizes the need for intense quality assurance efforts especially for new releases of
existing products, and where production applications depend on continued reliability. For
example, the reliability of Oracle’s flagship database products has widely been
recognized, and many leading analyst firms do not hesitate to recommend deployment of the
latest releases.

However, Oracle operates in a competitive software market, with
customers demanding new technologies for application development and deployment „at Web
speed”. Especially for new products, Oracle has to balance the need for quality
assurance against the requirements of getting products to market. To be competitive even
new products must be held to a high standard of quality, while recognizing the importance
of time-to-market. Oracle introduced significant new functionality like the JServer Java
VM in Oracle8i with outstanding functionality, scalability and reliability. Some new
products and technologies like Oracle’s Internet File System (iFS), which is now in beta
testing, have been delayed to ensure acceptable quality. In short, Oracle Corporation is
committed to excellent product quality while delivering the most competitive products on
the market in a timely fashion.

The recognized quality and reliability of the Oracle database products
requires significant effort and resources. The testing group has grown dramatically in
recent years, with the addition of significant testing hardware for testing and the hiring
of additional quality engineers to increase the ratio of testers to developers. For each
recent database release, the test suites by have grown by 30-40%, substantially more than
the product itself. These tests include a combination of regression tests, code coverage
tests, stress tests, customer application tests and internal production use before
release. Every night, thousands of regression tests are run on the software under
development. The methodologies used by Server Technologies for building, configuring,
integrating and testing the database product are continually being improved and applied to
additional components such as iFS.

Many of the comments from the IOUC survey addressed issues concerning
Oracle tools. Oracle aggressively migrated all of its applications and tools to a
Web-based architecture, and it has done so while preserving customers’ investments in
existing applications, which easily can be re-deployed to the Web. This strategy allows
customers to rapidly gain the benefits of Internet computing without significant
redevelopment cost. However, based on the IOUC survey, recent releases of Developer and
Designer have fallen short of users’ quality expectations. Oracle is fully committed to
improving the quality of these products in upcoming releases.

For example, Oracle has expanded by 20% the size of the teams devoted
to tools QA, product management installation and defect diagnosis and resolution. The
tools development group has also launched many specific product quality initiatives.
Before release, the Developer product now must successfully upgrade Oracle Applications
11i, involving tests of over 2400 reports and forms and millions of lines of PL/SQL code.
These tests are being expanded, and the Forms module in the pre-alpha release of Developer
7.0 has already achieved a 99% upgrade of Oracle Applications. The tools development group
applies similar tests to Developer patches. Additionally during the pre-release process,
several significant customer-provided applications are tested, including upgrade and
runtime certification of applications from major telecommunications, manufacturing and
insurance companies. Oracle continues to solicit representative applications from
customers to be included in upgrade and regression testing process.

The tools development group has also broadened internal visibility of
its product test plans, and has created a team to analyze all critical bugs and enhance
the test plans as a result. Bug resolution has been improved, with revisions to the bug
prioritization system and significant additional resources, resulting in a 50% decline in
the bug backlog over the past year. In Developer Release 6i, a new installation procedure
will automatically install and configure the run-time servers for Forms, Reports and
WebDB, greatly simplifying the process. Also, beginning in late 1999, Oracle introduced
the Tools CD Pack, which facilitates a quick, integrated and simple installation for all
Tools products certified to work together into a single Oracle_Home. Similar CD packs are
available for Oracle’s ERP and CRM Applications products and the Database Server
products.

Oracle recognizes with regret that there were serious quality issues
with Oracle Designer 2.1, which was introduced nearly 2 years ago. Oracle rapidly released
Developer 2.1.1 and 2.1.2, designed to fix problems in the initial release, and began a
focused bug-fixing effort and made changes in development processes with an eye to
improving quality. The Designer development group adopted the procedure used by the
database team to release tightly managed patch sets aimed at rapid fixes for the most
important problems. Two such patch sets have been released for Designer 6, with a third
release due in March 2000. These patch sets are coordinated with corresponding Developer
releases and have significantly reduced the number of outstanding bugs in recent releases.

Internal quality metrics and anecdotal evidence indicates that quality
has improved significantly, but much more is being done to raise the quality of the
Designer product. The responsible development group has reorganized and increased its
testing resources, so that so there is more testing being done earlier, with testers and
developers working closely to shorten cycle times and ensure comprehensive coverage of the
functionality. Qualified members of other Oracle organizations such as support, consulting
and education are now working with the Designer development team in an expert testing
program, providing in depth testing in focussed areas, and a „limited production”
program has been introduced to encourage heavy-duty testing in real-world environments.
Oracle has established clear quality targets for Designer6i such as closure of all high
priority bugs. Designer6i will not be released until these goals are met.

The curriculum development training function has moved into product
development organizations company-wide. This means better quality training materials, and
has allowed us to concentrate more on training of support personnel. Support analysts in
each support center have already been trained on Designer6i. The involvement of support
personnel in the expert testing program not only improves product quality, but also helps
ensure the ability and readiness of the support organization to support customers using
the new release.

Oracle Corporation sponsors Customer Advisory Boards (CABs) for major
products. These advisory boards offer customers the opportunity to engage with Oracle
development and product management groups, to provide feedback on product quality,
enhancements and future production directions. Participating customers are required to
actively beta test new releases to help ensure product quality, and (quite frankly) to
establish a visible, high profile customer reference base for each imminent release. The
CAB for Oracle Forms Server was established in January 2000.

Oracle Corporation continues to invest heavily to measure and improve
the quality of all of its products, including basic reliability, ease of use, integration,
and performance. Enhanced coordination of development across product groups and the use of
a standard Web-based architecture are beginning to show results in terms of reduced time
to market, improved product quality and integration, and a better user experience.
Improved product quality has resulted from earlier testing of tools and applications on
the latest releases of the database and lower layers of technology, along with automated
porting processes and the frequent updating of CD packs containing well-tested product
configurations.

Oracle is eager for customers to know about its commitment to product
quality. At the November 1999 OpenWorld Conference, Ray Lane, Oracle’s President and
COO, specifically addressed the results of the 1999 IOUC user survey. This tradition will
be continued each year, along with regular dialogue with IOUC board members about product
quality and other customer satisfaction issues. In line with Oracle’s strategic
commitment to fully becoming an e-business, Oracle makes bug information available to
supported customers online via Metalink, so users can directly access specific bug reports
and assess product quality. In addition, Oracle provides a vast library of sample
applications and technology usage information about Oracle technology via the Oracle
Technology Network (http://technet.oracle.com) and
Oracle AppsNet (http://appsnet.oracle.com).

Oracle recognizes the value of customer satisfaction, and the
importance of basic product quality as a key determinant of customer satisfaction.

4. PRODUCT RELEASE PROCESS

It is clear from the answers given in the survey that there is a
perception that Product Development, Quality and Support go hand-in-hand.

The IOUC suggests that a strengthening of the Product Release Process
(PRP) would address some of the issues raised. We understood from answers given by Oracle
in previous IOUC Special Reports (see Vol. 3, No. 2, Oct 1999), that PRP was meant to tie
various functions together.

We were advised in April 1998 (Vol 2 No 2 – Product Quality Page 3)
that the PRP addressed the issues of quality from design to beta programmes. We have now
discovered that the PRP is merely a checklist to ensure that all divisions in Oracle are
ready for the release of the product. The PRP simply requires Test Plans and Quality
Testing it does not stipulate the requirements of those tests. Vol.1 No.1 contained
detailed descriptions of Oracle’s plans for ensuring Quality Assurance throughout the
organisation – what has happened since 1996?

Clearly a rigorous PRP regime would be beneficial for the products the
customer receives, although we recognise that it could cause some lengthening of the
development phase.

The following questions thus arise;

  1. What products have been released after going through PRP?
  2. Is it intended that it will eventually be mandatory for all products to go through PRP?
  3. Who is (or will be) responsible for enforcement?
  4. Is PRP intended as a product life-cycle methodology? If the whole product life cycle is
    not covered, what plans are in place to correct that?
  5. Who determines the release date for products (development or marketing)?

ORACLE’S RESPONSE:

The Product Release Process (PRP) provides a systematic approach to
managing Oracle’s products from conception to product release. Its primary purpose is to
facilitate product readiness through teamwork and communication. While a byproduct of the
process is improved product quality, it is not intended to be a direct part of the Product
Release Process programme. The PRP is a framework of proven Oracle best practices to
facilitate release readiness across the company. Product groups have product specific
development processes and requirements, and the PRP is designed to be customized to each
product’s release effort. PRP defines the requirements necessary to insure a coordinated
product release or as we refer to it, ‚release readiness’.

Most products have adopted the PRP framework. While some products have
release cycles greater than 12 months (i.e., Apps 11i), products released to market in the
past 12 months utilizing the PRP include:

  • Oracle8I (8.1.5, 8.1.6 )
  • CRM 3i
  • WebDB
  • Developer
  • Designer
  • JDeveloper
  • Discoverer
  • Applications Server
  • Energy Upstream
  • Public Sector Applications
  • Student Systems
  • Message Broker
  • OEM
  • CPG
  • Applications System Bundle
  • 8i Appliance
  • Financial Services Applications
  1. Is it intended that it will eventually be mandatory for all products to go through PRP?
    While the goal is for every product to adopt the PRP framework, there are cases when the
    PRP simply does not apply (HP Openview, Portal-to-Go, etc.).

  2. Who is (or will be) responsible for enforcement?
    Once a product engages in the process, the PRP team is responsible for determining the
    necessary input and feedback mechanisms.

  3. Is PRP intended as a product life-cycle methodology? If the whole product life cycle is
    not covered, what plans are in place to correct that?
    The PRP is a framework to drive release readiness within Oracle. Various tools have been
    implemented to assist the process, and both the methodology and tools undergo continual
    refinement. For example, because each product group customizes the PRP to its release
    effort, tools allowing teams to store and access release data in a single repository allow
    customization of deliverables/milestones. PRP does offer guidelines for post release
    phases and deliverables.

  4. Who determines the release date for products (development or marketing)?
    Release dates are based on a collective decision within the product groups. Products have
    defined roadmaps targeting regularly scheduled releases. However there are many internal
    and external factors (business and technological objectives) that may impact the release
    date.

5. PRICING

Pricing was the second most likely topic to be raised with a Senior
Executive.

And it is clear from the Free Format answers that there is very
considerable concern, confusion and frustration about Pricing policy.

When answering the question, „On your last purchase from
Oracle, what type of method was used to determine pricing?” (Q16), nearly 95% responded
that it had been „Per Concurrent Device.” However, there was a significant Gap between
Importance of this method of pricing and Satisfaction with the outcome, as indicated in
the answers to Q17.

Importance Satisfaction Gap
Pricing (Per Concurrent Device) 7.71 5.89 1.82
  1. It is understood that since this survey was conducted, Oracle has standardised on a
    method of Pricing. Does it apply globally?
  2. How are customers being informed of the new pricing structure?
  3. Does the method disadvantage customers (e.g. price increases without increased
    functionality