!!!HSE GCII 2022 methodology 25.05.2022

AGENDA
Challenges of measuring innovation at the city level
Theoretical framework for measuring innovations used by the Russian Cluster Observatory
HSE Global Cities Innovation Index 2020
System of indicators
Sample of cities
HSE Global Cities Innovation Index 2022
Amended system of indicators
Changing the approach to city sampling
Approaches to identifying agglomerations
Publication and patent analysis methodology
Q&A session
Questions for discussion

CHALLENGES OF MEASURING INNOVATION AT THE CITY LEVEL
Lack of reliable data sources for international comparisons of cities
by their innovation development
The use of a small number of indicators reflecting the results of scientific activity (patents and publications), or reliance on unverifiable expert assessments and surveys
Result: lack of a comprehensive vision of the objective comparative advantages of innovation centers opportunities to significantly improve the quality of strategic planning and offer more specific tactical solutions for city managers
No unanimous position among researchers on the content and measurement methods of cities' innovation development trends
Formation of isolated ratings for technological and digital development, creative potential and infrastructure
No unified concept among countries on what constitutes a city/agglomeration
Comparison of "convenient" cities (London, New York, Tokyo, Paris, etc.), ignoring other real competitors in the field of innovation (i.e. Silicon Valley)

OUR APPROACH IS BASED ON THE CONCEPT OF THE SUPERSTAR ECONOMY
Relatively small numbers of people earn enormous amounts of money (Rosen, 1981). These people may be called A-list (Caves, 2000) or superstars
2. Superstar people tend to be concentrated in superstar countries
3. Superstar countries enjoy most of the gains from progress, with other countries being increasingly left behind (Korinek and Xuan, 2018)
4. The rise of human superstars is just beginning (Korinek and Xuan, 2018)

CITY INNOVATIONS SHOULD BE MEASURED USING AN ECOSYSTEM APPROACH
Innovation ecosystem of a global city involves creators of products and technologies attracted by recognized leaders of the knowledge economy (superstars)
Last but not least, there is an advanced infrastructure and friendly environment in such cities

HSE GCII 2020

Transparency – use of open international databases
Verifiability – refusal to use "internal“ data of city administrations on various aspects of innovative development, inaccessible to a wide range of users
Comparability – the data used allows for the most objective comparison of cities
Objectivity – rejection of opinion polls or expert interviews
Technological
Development
Creative
Industries
Urban
Environment
Fortune Global 500
Innovation 1000
StartupBlink
Crunchbase
QS
THE
ARWU
Web of Science
PatStat Global
FARFETCH
Fashion United
IMDb
Steam
if
Provoke Media
World Architecture Festival (WAF)
Nomad List
STC Database
PassportEuromonitor
Numbeo
World Metro Database
WiFi Map
OpenFlights
World Value Survey
Effie Awards
Spotify
The Game Awards
Artprice
Pritzker Prize
TripAdvisor
DATA COLLECTION PRINCIPLES
Spotify
Cannes Lions
Wikipedia
Reddot

SYSTEM OF INDICATORS
Overall HSE GCII score
Creative Industries
Urban Environment
Technological Development
Technology companies
Startups and venture capital
Universities and R&D organizations
Productivity of innovative class
Innovation infrastructure
Fashion industry
Film industry
Electronic game industry
Advertising and PR
Industrial design and architecture
Arts and culture
Cost of doing business
Cost of living
Transport infrastructure and mobility
Digital infrastructure and services
Safety and security
Tourist appeal
Ecology and climate comfort
Inclusivity
Values
120 indicators

SAMPLE: 36 CITIES THAT ARE THE WORLD'S LEADING CENTERS OF INNOVATION
SELECTION CRITERIA
Leading cities in the key developed and developing countries in terms of the number of patents and publications
Presence of a city in international rankings on relevant topics
Completeness of data on a city in research information sources (>90% of indicators)

METHODOLOGY
Each indicator’s absolute score was normalized using formula (1) or formula (2), depending on the indicator’s effect on the overall GCII index
a higher value of the indicator corresponds
to a greater innovative attractiveness
a higher value of the indicator corresponds
to a lower innovative attractiveness
(1)
(2)
xi is a city’s indicator score
xmax is the highest indicator score for all cities in the sample
xmin is the lowest indicator score for all cities in the sample
i is the number of a city
section score
sub-index score
HSE GCII score
The values of sections, sub-indices and the integral HSE GCII are calculated through indicators

HSE GCII: 2020 RESULTS

NOT JUST A RANKING OF CITIES, BUT A TOOL FOR POLICYMAKERS

BEST PRACTICE CASES
Presented only in the Russian version of the HSE GCII 2020

HSE GCII 2022

CURRENT CHANGES
Improved system of indicators
2. More reasonable approach to city sampling
3. Transition from cities to agglomerations
4. Formation of long and short lists of the HSE GCII (under discussion)

CHANGES IN THE SYSTEM OF INDICATORS: STATISTICAL AUDIT
Coefficients of kurtosis and skewness
Sensitivity of the integral rank of the city to changes in individual indicators
Correlations between indicators, sub-indices and the integral HSE GCII
Revenues of largest companies
Domestic faculty staff
International faculty staff
Fashion designers from Big 4 Fashion weeks
Highest-rated films (critics)
Gender balance
A number of "Values" indicators
The indicators excluded
HSE GCII 2020 database statistical audit
Note: Sections 3.5 "Safety", 3.7 "Ecology and climate comfort", 3.9 "Values", as well as a number of indicators of the "Inclusion" section are also subjects to exclusion. The decision on these sections will be made after additional statistical analysis of the updated database

CHANGES IN THE SYSTEM OF INDICATORS: EXPERT DISCUSSIONS
Based on the results of discussions with experts (creative communities, researchers, etc.), the following changes have been made:
New source for "Technology companies" section (R&D scoreboard 2500 instead of Fortune Global 500 and Global Innovation 1000)
New indicator "R&D Intensity of innovative companies"
The "Unicorns" indicator uses an additional data source – CBInsights (in addition to Crunchbase)
In the "Creative industries" sub-index new sections "Sound recording and performing arts" and "Literature“ were added. Section "Architecture and Industrial Design" was divided into two: "Architecture" and "Industrial Design"

SYSTEM OF INDICATORS
Overall HSE GCII score
Creative Industries
Urban Environment
Technological Development
Technology companies
Startups and venture capital
Universities and R&D organizations
Productivity of innovative class
Innovation infrastructure
Fashion industry
Film industry
Electronic game industry
Advertising and PR
Industrial design
Arts and culture
Cost of doing business
Cost of living
Transport infrastructure and mobility
Digital infrastructure and services
Safety and security
Tourist appeal
Ecology and climate comfort
Inclusivity
Values
Architecture
Sound recording and performing arts
Literature
Excluded
Included

SELECTION OF INDICATORS ON THE BASIS OF WHICH THE RESEARCH SAMPLE
IS FORMED
A set of HSE GCII 2022 indicators collected for all possible settlements:
Largest companies (2 500 observations)
Unicorns (1 302 observations)
Leading universities (2 051 observations)
Highly cited researchers (6 332 observations)
Nobel Prize laureates and Fields Medal winners (384 observations)
Leading business schools (185 observations)
Technology and science parks (306 observations)
All "Creative Industries" sub-index indicators. For example, "Fashion brands" (2 589 observations), "Highest-rated films" (913 observations), "Cybersport Tournaments" (343 observations), "Most traded living artists" (206 observations)

SAMPLE
Collection of data on the selected indicators identified
2 769 unique settlements in 134 countries
OECD countries
(34 countries with 1 997 settlements)
Non-OECD countries
(100 countries with 772 settlements)
1 753 settlements
Functional Urban Areas (FUAs)
Individual approach

METHODOLOGY (UNDER DISCUSSION)
Calculation of the integral index through sub-indices
Defining weights for sub-indices
Overall HSE GCII score
Creative Industries
sub-index
Urban Environment sub-index
Technological Development sub-index
40%
40%
20%
FOR SHORT-LIST CITIES (204 settlements)
FOR LONG-LIST CITIES (1753 settlements)
Overall HSE GCII score
50%
50%
Technological Development sub-index
Creative Industries
sub-index

PUBLICATION ACTIVITY ANALYSIS: BASIC PRINCIPLES
Database for analysis: Scopus
Time period: 2010-2021
Document types included: articles, reviews, proceeding papers, books, book chapters, letters, notes
In most cases we run the search of names of cities, towns, villages (and other settlements) included in the studied sample of agglomerations in AFFILCITY() search field in Scopus (i.e. among city names automatically detected by Scopus).
Example for Beijing:
PUBYEAR > 2009 AND PUBYEAR < 2022
AND DOCTYPE ( ar OR re OR cp OR ch OR bk OR le OR no )
AND AFFILCITY ( "Beijing" OR Peking OR "Langfang" OR "Zhuozhou" )
AND AFFILCOUNTRY ( China )

SEARCH FOR PUBLICATIONS OF A SPECIFIC CITY: HOW DOES IT WORK
Take Beijing as example:
Scopus Query search: PUBYEAR > 2009 AND PUBYEAR < 2022 AND DOCTYPE ( ar OR re OR cp OR ch OR bk OR le OR no ) AND AFFILCITY ( "Beijing" OR Peking OR "Langfang" OR "Zhuozhou" ) AND AFFILCOUNTRY ( China )
Bai Yanan marked the affiliation as: University of Chinese Academy of Sciences, Beijing, 100049, China
Beijing name of city was correctly determined by AFFILCITY() Scopus search field and this publication was counted as publication if Beijing.
We do not know if Bai Yanan is really working in University of Chinese Academy of Sciences in Beijing. Possibly affiliation with this university is only formal for this author.
Nevertheless we count all publications where Beijing name of city was determined by AFFILCITY() Scopus search field as publications of Beijing.
Screenshot of a publication affiliated with Beijing:

PUBLICATION ACTIVITY ANALYSIS: METHODOLOGICAL ASPECT OF SEARCH OF CITIES
When we have two cities with the same names located in different countries (e.g. Cambridge, Massachusetts, USA and Cambridge, East of England, UK) we use AFFIL() search field and country name restriction.
When we have two cities with the same names located in different states of the USA (e.g. Wilmoington, Massachusetts, USA (Boston agglomeration) and Wilmington, Delaware, USA (Philadelphia agglomeration)) we use AFFIL() search field and state name restriction.
See the example of Boston and Philadelphia agglomerations:
Pubyear > 2009 and Pubyear < 2022 and doctype(ar OR re OR cp OR ch or bk or le OR no) And ( Affilcity( "Waltham" OR "Medford" OR "Andover" OR "Bedford" OR "Chestnut Hill" OR "Framingham" OR "Watertown" OR "Billerica" OR "Beverly" OR "Lowell" OR "Somerville" OR "Walpole" OR "Wellesley" OR "Acton" OR "Braintree" OR "Chelmsford" OR "Danvers" OR "Maynard" OR "Natick" OR "Newton" OR "North Reading" OR "Westford" OR "Woburn" ) OR Affil( ("Cambridge" OR "Boston" OR "Marlborough") and (U.S. OR US or USA or “United States”) ) OR Affil("Wilmington" and (MA or Massachusetts) ) ) AND affilcountry( “United States” )
Pubyear > 2009 and Pubyear < 2022 and doctype(ar OR re OR cp OR ch or bk or le OR no) And( affilcity( "Philadelphia" OR "Princeton" OR "Malvern" OR "Camden" OR "Exton" OR "King of Prussia" OR "New Castle" OR "Swarthmore" OR "Ambler" OR "Pennsauken" OR "Wynnewood" OR "Audubon" OR "Collegeville" OR "Conshohocken" OR "Ewing Township" OR "Glen Mills" OR "Plymouth Meeting" OR "Radnor" OR "Yardley" OR "Pottsgrove" OR "Solebury Township" OR "Bryn Mawr" OR "Glassboro" OR "Haverford") OR affil("Wilmington" and (DE or Delaware)) OR affil(Wayne and (PA or Pennsylvania))) AND affilcountry(“United states” )
We take all variants of names of key cities of agglomeration on English and national languages (as well as transliterated names)
Example of Munich, Germany:
Pubyear > 2009 and Pubyear < 2022 and doctype(ar OR re OR cp OR ch or bk or le OR no) And affilcity("Munich" OR München OR Munchen OR Minga OR "Garching" OR "Gilching" OR "Unterföhring" OR "Landsberg am Lech" OR "Martinsried" OR "Neubiberg" OR "Planegg" OR "Stockdorf" OR "Taufkirchen" OR "Oberhaching" OR "Olching" ) AND affilcountry(Germany )

PATENT ANALYSIS: SEARCHING STRATEGIES
Database: PatStat Global
Time period: 2010-2021
We counted patent applications which, unlike patents granted, provide an up-to-date
picture of the current situation without significant time lag.
Step 1. Searching by city
City search in the address and in the name of the organization, excluding double counting
Country code restriction
Step 2. Searching by alternative city names
person_address like '%Hangzhou%' or '%Hangchow%' or '%Hángzhōu%'

PATENT ANALYSIS: ASSESSMENT OF THE REPRESENTATIVENESS OF THE DATABASE
Calculation of target benchmarks for cities, clusters (Global Innovation Index databases), and relevant countries (World Intellectual Property Organization (WIPO) database) using available data.
Comparison of results with the database for the HSE Global Cities Innovation Index 2020.

Q&A