Best International Support Sites: Scoring Criteria

A. Overall Usability and Quality

Based on your first impressions of the site and your subsequent exploration of key features, please assign scores for the following five criteria:

1. From the organization’s main support page, how easy is it to find localized support information?

(4.0)	Links to localized support were obvious and intelligible without knowing another language. Individual local product pages were easy to find
(2.0)	Access to localized content was logical, but required extraneous steps (e.g., extra clicks, reselecting the language, or logging in more than once).
(0.0)	Localized support content was not easily accessible.

2. How would you rate the coverage and timeliness of localized content.?

(4.0)	Excellent. All relevant features from the source site are functional, easily accessed and well maintained. All features work as expected. The latest information is available.
(2.0)	Average. Not all relevant features from the source site are functional, easily accessed or well maintained. Some features do not work as expected, but are usable. Recent information may be lacking.
(0.0)	Poor. Many features are missing, inaccessible or out of date. Many features do not work as expected and are unusable.

3. How would you rate the site’s overall localization quality?

(4.0)	Comparable to a good-quality locally developed site. Text and graphics are skillfully localized and appropriate for the market/country for which they are intended.
(2.0)	Average. Text and graphics are acceptable in the target market, but are obviously localized from another market.
(0.0)	Poor. Text and graphics are unacceptable in the target market or have extensive errors specific to the target market.

4. How well does the site do in meeting local needs?

(4.0)	Excellent. Products and services addressed in the site pertain to the local market and the site is useful to local users, not just to users in the organization’s home country. The site gives the feeling that it is truly engaged with local requirements.
(2.0)	Average. The site is an adaptation of the source-language site that is not particularly adapted to the local market. It may address products and services not available in the target market but may not be fully engaged with local market needs and trends.
(0.0)	Poor. It fails to meet the needs of the local market. Needed features and information are lacking or incorrect for the target market. (E.g., address forms cannot be used for local addresses.)

5. How easy is to find locale-specific contact information such as phone numbers, e-mail addresses, mailing addresses, and locations of service centers?

(4.0)	Very easy. Links to locale-specific contact information are readily available and clearly appropriate to the target market.
(2.0)	Somewhat difficult. Links take some effort to find or are not clearly appropriate for the target market (e.g., they are for regional centers that may not cover the local language or market in specific).
(0.0)	Very difficult. There are no links for locale-specific contact or such links resolve to support outside of the appropriate markets.

B. Knowledgebase and Search Implementation

After exploring the site’s localized content, comparing it to the source language version, and trying a few knowledgebase queries, please assign scores to the following five criteria:

1. What is your impression of the site’s overall localized knowledgebase content (tech notes, articles, tutorials, FAQs, etc.)?

(4.0)	Comprehensive, in-depth collection of several different kinds of content. Most relevant information available in the source language is available in the target language and any topics important to the local market are covered in detail.
(2.0)	Coverage is basic, with high-priority materials localized. Topics specific to the local market are addressed but may not be addressed in detail.
(0.0)	Coverage is minimal, with extensive coverage gaps and missing essential information that is available on the source site. Topics specific to the local market are not addressed.

2. What is your impression of how the localized search function displays results?

(4.0)	Results are clearly ranked by appropriateness with respect to my search terms and searches can be refined easily. Titles and summaries are clear and in the local language. If results point to documents in the source locale that are not available in the target locale, options are available to receive them in the locale language (e.g., using machine translation or by allowing users to request translation).
(2.0)	Results are returned in the local language but lack contextual information needed to evaluate their suitability or their ranking is unintuitive. If results are returned from the source locale, there is no obvious provision to have them localized.
(0.0)	Search results are not ranked by relevance and summaries and titles are either irrelevant or display in the wrong language.

3. What is your impression of FAQs (or simlar sections)?

(4.0)	The answers are well written, answer “real” questions, and address locale-specific issues or requirements in addition to general product issues. They cover a wide variety of topics relevant to users of the website. Any supporting materials (e.g., screen shots) included in answers are localized
(2.0)	The answers exist but do not address only general and not locale-specific issues, or address topics haphazardly or incorrectly with regard to localized issues. Answers may include unlocalized supporting materials (e.g., the answer may be in French, but screen shots are in English).
(0.0)	There is no local-language FAQ or is is hard to find or incomplete.

4. How would you describe the writing and localization of the knowledgebase content?

(4.0)	Well written and fully localized, with clear understanding of local language methods of information presentation and understanding. All links to other content work and there are no major gaps in coverage where links point to unlocalized content. Coverage in the target language is comparable to that of the source language.
(2.0)	Generally accurate and well localized, but information presentation may be odd for the target language or some portions of the knoweldgebase are unlocalized.
(0.0)	Information is poorly translated, obviously incomplete, has many language mistakes, or is very unclear for the target audience.

5. How well does the site do in soliciting feedback on localized knowledgebase materials?

(4.0)	There is a clear path to submit local-language suggestions and quality assessments applicable to the local language (rather than to the topic as a whole). Confirmation of receipt it provided in the appropriate language.
(2.0)	Feedback mechanisms are limited to the topic as a whole rather than to localized versions, or methods for submitting locale feedback are limited (e.g., only a generic e-mail address like feedback@xyz.com is provided for feedback). Confirmation of feedback is not localized and there is no way to be sure that it has been received and understood properly.
(0.0)	There are either no feedback mechanisms or they either ignore localized content versions or redirect to pages in an incorrect language.

C. Interactive Features

Please assign scores for the following five criteria:

1. Does the site invite users to sign up for localized e-mail alerts, newsletters, tips, etc., and does it appear that the company actually sends out localized material?

(4.0)	Yes, the company invites users to sign up for localized content and sends out appropriately localized materials.
(2.0)	Yes, the company invites users to sign up for localized content but actual content is limited or not of the same quality as what someone signing up for material from in the source language would receive.
(0.0)	There are no options to sign up for localized content from the company.

2. What options are available for personalized support in the local language?

(4.0)	Users can log new cases in their own language and check the status of pending cases.
(2.0)	Users can submit questions in their own language and receive knowledgeable answers back in a reasonable amount of time.
(0.0)	There is no option for localized case management or personal interaction via e-mail in the local language.

3. How integrated is the support site for the local market?

(4.0)	Site serves as a central portal for all post-sales customer interactions in the local market, including support, training, consulting, and user community.
(2.0)	The site serves as a portal for multiple services but does not support all post-sales customer activities and requires customers to go outside the site (e.g., to local physical sales points) for essential needs.
(0.0)	The services are limited to knowledgbases and downloads

4. If the localized site fails to resolve a problem, what local-language options do (free or fee-based) do users have and how do they compare to options in the source language?

(4.0)	Most options on the source site are available and local-language phone is an option.
(2.0)	Fewer options than the source site, primarily e-mail, without live local-language phone.
(0.0)	There are no local-language escalation options.

5. How well does the site do in providing a localized multimodal interactive user experience?

(4.0)	A variety of options, including chat, online discussion forums, interactive webinars, and other multimodal opportunities for interaction are available in the local language, attended by knowledgeable employees, and are available during appropriate hours for the local market.
(2.0)	Local-language options are limited to a few options. Hours may not be ideal for the local market (e.g., Spanish chat is available, but not during European business hours).
(0.0)	No multimodal interaction or options are not available in the local language.

D. Customer Experience

Please assign scores for the following five criteria:

1. How good is the site at provided a localized and personalized support experience (e.g., a personal home page)?

(4.0)	Very good. Personal home pages are localized well and provide easy access to localized product support pages and related materials and automatically adapt to customer needs based on viewing history or other criteria.
(2.0)	Reasonably good. Localization is acceptable but pages may be based strictly on product ownership without dynamic content.
(0.0)	There are no personal home pages or views.

2. How well does the site support finding and obtaining appropriate local-language upgrades, downloads, and drivers?

(4.0)	Easy. The site automatically selects the appropriate language version (or provides multiple-language downloads that work with any language-version of the product) without need to select between multiple language versions. Any auto-detection tools work for localized products.
(2.0)	Moderately easy. Materials are easy to access but must be chosen from amongst multiple language versions of the same product rather than a prioritized list of relevant materials.
(0.0)	Difficult. Local-language versions do not exist, are difficult to find and access, or must be specifically requested rather than downloaded.

3. How good is support in the target locale for various customer learning styles?

(4.0)	The target locale has many localized options, including ones appropriate for the target audience.
(2.0)	The source locale has many options that are not represented in the target locale or neither source nor target have many options.
(0.0)	The localized site seems to have a “one size fits all” approach.

4. Is the site optimized to serve various customer segments (e.g., dealers, novices, developers), including specific regions or languages?

(4.0)	Offers specific areas for different customer types, including ones for individual regions or languages.
(2.0)	Some content is segment-specific, but is not separated from general content, or does not address regional or language needs.
(0.0)	Site seems to have a “one size fits all” approach

5. How would you describe the site’s use of multilingual technologies?

(4.0)	Performance is excellent and technologies used support multiple languages in an intuitive manner.
(2.0)	Performance is acceptable and local languages are supported adequately although some features may be limited.
(0.0)	The technologies chosen have serious problems in supporting specific languages. Characters, addresses, etc. may be garbled.

E. The Localization Process (Essay)

Based on your review of the localization process essay and your experience with the site, please assign scores for the following five questions:

1. How ambitious are the company’s plans and processes for improving localized support and ensuring that local needs are met in a timely manner?

(4.0)	The plans provide a clear and comprehensive method or process for ensuring that international customers receive up-to-date quality support on par with support in the home market. Technical advances in the source site will be reflected immediately (or with minimal delay) in the localized site.
(2.0)	The plans ensure that support sites will be updated after a delay, but not all features will be available in target languages.
(0.0)	Local-language support sites are developed separately from the source site and are not synchronized. Technical developments in the source site are unlikely to be reflected in the localized site.

2. In terms of the company’s available resources, how ambitious was the challenge of integrating, improving, or extending localized support over the last year?

(4.0)	Intense effort, pushed the limits of available resources and extended locale support considerably.
(2.0)	A separate development project, well-managed but fairly low risk.
(0.0)	Business as usual, not much extra efforts of resource commitment.

3. How well did the site’s developers succeed in providing full-spectrum support for local-language needs?

(4.0)	They succeeded in supporting the needs of local-language audiences on a par with or better than organizations based in the local market
(2.0)	They succeeded in meeting basic support needs, but locally-based companies have a clear advantage in the support arena.
(0.0)	Support is well below par for this market when compared to other similar companies (both international and local).

4. How did the site developers know that their localized sites meet quality and market needs?

(4.0)	They analyzed their site with a formal process including work with end-users from the local market.
(2.0)	They relied on anecdotal feedback or on evaluation by staff in the target market rather than end-users.
(0.0)	They have no way of assessing success or failure.

5. How would you describe the localization “lessons learned” from this process?

(4.0)	Lessones are insightful, strategic, and applicable to a broad range of locales and situations. Other organizations can learn valuable lessons from this example.
(2.0)	Lessons are mostly technical and apply only to the company itself. Other organizations using different technical platforms may not learn much from this example.
(0.0)	Few insights were gained from the site-development experience. Others are unlikely to learn from the example.