An open database of productivity in Vietnam's social sciences and humanities for public use

This study presents a description of an open database on scientific output of Vietnamese researchers in social sciences and humanities, one that corrects for the shortcomings in current research publication databases such as data duplication, slow update, and a substantial cost of doing science. Here, using scientists’ self-reports, open online sources and cross-checking with Scopus database, we introduce a manual system and its semi-automated version of the database on the profiles of 657 Vietnamese researchers in social sciences and humanities who have published in Scopus-indexed journals from 2008 to 2018. The final system also records 973 foreign co-authors, 1,289 papers, and 789 affiliations. The data collection method, highly applicable for other sources, could be replicated in other developing countries while its content be used in cross-section, multivariate, and network data analyses. The open database is expected to help Vietnam revamp its research capacity and meet the public demand for greater transparency in science management.

To achieve the goals, the search covers only Vietnamese researchers in SS&H of Vietnamese nationality that meet at least one of the following criteria: • They are affiliated with an organization in Vietnam; OR • They have published at least one paper about Vietnam or use data collected in Vietnam related to SS&H topics.
The search is further confined to Vietnamese authors who have published in Scopus-indexed scientific journals. It is important to note that the method could in principle cover publications indexed in the WoS, MathSciNet, PubMed and other reliable scientific databases. For comparison purpose, Scopus indexed about 22,600 titles 20 , which is almost twice as many as its counterpart WoS 21 . Given the project aims to serve Vietnamese science policymakers, we take into account the fact that Scopus is one of many scholarly databases used by the Vietnamese government to judge academic credentials 22 . Specifically, in a governmental decision, the Vietnam National Foundation for Science & Technology Development (henceforth referred to as NAFOSTED), Vietnam's leading funder for science and technology research, has provided a list of prestigious international and national journals in the field of SS&H, which includes being indexed in Scopus as a criteria 23 . This is also a common practice in various countries including the United States, Spain, and Russia 24-26 as well as for highly influential rankings such as the Times Highers Education 27,28 .
Based on these basic principles, next we will delve into the manual data collection system, its procedure and shortcomings that prompt the need for the semi-automatic system.

NVSS Manual System
The manual process of data collection and verification was carried out from 1 st February 2017 to 15 th July 2017, which resulted in the creation of the Network of Vietnamese Social Scientists (NVSS) dataset. NVSS contains 412 science profiles for 412 distinct Vietnamese researchers in social sciences and humanities who have published in Scopus-indexed journals. An example of these first science profiles could be found in Data Citation 1. Phase 3: Constructing Cross-section and Network Datasets from all the science profiles. Figure 1. Project conceptualization. The project, from data collection to datasets construction, consists of three phases. Phase 1 is about identifying Vietnamese social scientists who have international publications and cross-checking among various data sources. Phase 2 is about creating a personal science profile for each author which includes information on the author's scientific output and demographic characteristics. Phase 3 is about constructing cross-section and network datasets from all the science profiles.

Procedure
The first step of the data collection process was to access websites of research institutions in Vietnam to identify researchers who fit the above criteria. Then, based on their public CVs, we marked down the number of publications they have authored and their demographic information. Next, we cross-checked these newly gathered data with websites of journals, Google Scholars, Scimagojr, and Scopus to make sure the information claimed on the CVs was in fact accurate. The Scopus system, therefore, has only value for us to double-check by examining if a randomly chosen research item has been present in their indexing system. To ensure that the manual process covers as many eligible Vietnamese researcher as possible, we also looked at the references lists of the articles and experts' opinions, as well as used varied keywords ('Vietnamese economic development', 'Vietnamese history', 'Vietnamese culture', etc.), and other resources such as social media, online news outlets, to name a few. The experts are from organizations such as the State Council for Professor title of Vietnam; the Scientific Committees of NAFOSTED; other scientific boards of leading research institutions such as national universities; Vietnam Academy of Social Sciences; etc. or others with long-term experiences or high productivity in their respective disciplines. In the data collection stage, our team members would reach out to the experts for suggestions or confirmation of eligible researchers, then subject these suggestions to the rigorous cross-validation process.
The second step was to create a personal science profile for each Vietnamese author. Each said science profile corresponded to 13 lines of data (see Table 1). This process resulted in a clean, concise dataset of the most updated and complete profiles. We then contacted and invited the researchers to corroborate the profiles made by our team; the examples of some corroborated profiles could be found in Data Citation 1's Scientific Profiles (Examples) folder. A list of input names and explanations appears in Table 1 while their relationships are illustrated in Fig. 2.
The third step involved summarizing all the profiles into a master file. The example of the master file resulted from the manual system could be found in Ho et al. 2,9 .

Shortcomings
This manual method, albeit rigorous, faces two major shortcomings. First, the manual input of data is time-consuming and rigid, thus prone to human errors. The resulting dataset enables us to count how many publications each author has but lacks the capability for counting how many unique publications and journals exist in the entire database. This loophole excludes us from answering important questions such as how many new articles Vietnamese social scientists produce each year; or from generating data on international co-authorship network. Second, because the contribution-adjusted productivity ('cp') was computed manually, it would be immensely costly to switch to a different counting method such as the norm of all authors getting an equal share or the norm of first-last emphasis 29,30 .

SSHPA Semi-automatic system
The semi-automatic system, called Social Sciences & Humanities Peer Awards (SSHPA), was kicked off on 1 st December 2017 and wrapped up on 2 nd February 2018 to resolve problems posed in the manual Input name Variable name Explanation 10 Contribution-adjusted productivity cp: cp.sdc, cp.eq, cp.fl The measure of relative scientific output. When we calculate this measure manually, the absolute productivity is converted using the method of sequence-determines-credit (cp.sdc) [29][30][31] . Then, using the semi-automatic system, we added two more measures: equal contribution for all (cp.eq) and first-last-author emphasis (cp.fl) 29 process. The purpose was to have a system capable of: (i) validating the quality of data previously collected, and (ii) making our database more flexible, less time-consuming to construct, and less prone to human errors. The semi-automated process also enables us to cover as close as possible to the actual number of eligible Vietnamese social scientists. For a brief overview of the distribution by sex, there were 262 female (39.88%) and 391 male researchers (59.51%), with four left unknown. Table 2 shows the descriptive statistics for continuous variables used in the SSHPA system. Other datasets related to these statistics can also be viewed in Data Citation 1's Extracted and Computed Data's table.
System architecture The SSHPA system, accessible online at https://sshpa.com/, is structured in MS SQL Server 2012 and is indexed to search Fulltext to centralize the management process. Its architecture is organized according to Client-server architecture. The software Server is built using Net Core which provides the APIs connections and functional modules such as Data Search & Filters, Data Validation, Network Builder and Reports. In addition, SSHPA Client software is built with C# that connects the database server through REST API Interface, this is intended to provide the users with complete data-input and data-check functions. Similar to the manual data collection process, the first step was to search for profiles of Vietnamese social scientists fitting our criteria. As shown in Fig. 3, we collected the profiles provided by researchers and organizations then verified with other sources such as government websites, NAFOSTED's designated publications, journal websites, Scimagojr, Google Scholars, Scopus' freely accessible data, etc.
The verified data were then entered into the SSHPA database and put through automated quality assurance and quality control steps. SSHPA was also designed with an authorization system with three levels: admins, supervisors, and collectors. Collectors could only input and edit unapproved data. Supervisors could approve a data entry, however, once the data entry is approved as most complete and accurate by the judgement of the supervisors, it cannot be changed or removed by either the supervisors or the collectors. Only the admins could remove a data entry or unlock the approved data for changes. Hence, in each level of authorization, each person must be accountable for the accuracy and reliability of the data entered into the system. With the nature of being semi-automated, SSHPA was still prone to human errors; this authorization mechanism was a way to uncover the mistakes in a timely manner, and thus, minimizing the consequences.

Data structure
The data, once entered into our system, were organized in table structure in RDBMS.
We designate Article as the fundamental unit of SSHPA's data structure (center of Fig. 4), because: (i) an article's name is often long enough to reduce the odds of data duplication, and (ii) an article published on a journal's website will provide the other information such as authors, authors' affiliation,  Figure 2. The relationship among variables recorded in this study. A personal science profile consists of five groups of factors: scientific output, demographic factors, collaboration factors, fields of study and affiliation. Scientific output factors concern with total number of publications, solo publications, publications in leading (key) position, and contribution-adjusted productivity. Demographic factors include age, gender, regions, and career age. Collaboration factors concern with total number of collaborators, of domestic collaborators, and of foreign collaborators. Two other factors are fields of study and affiliations. publication year, and so on. This means all the other kinds of data: Author, Affiliation, Source, Publisher, Network, etc. are connected through Article. For example, the datArticle box and the datAuthor box are connected to each other through an intermediary, datArticleAuthor, which holds information that connects the authors with their publications such as: author IDs, article ID, order of the author(s), affiliations of the authors, etc. The datArticle box contains the relevant data on the articles or publications in the database: title, document type (proceedings or journal articles for example), publisher ID, journal ID, etc. The data are fed from other boxes which contain information on the publishers (lstPulisher), the sources (journals or proceedings of conferences or books) of the articles (lstSource), the citation information (lstCitation), or the document type (lstDocumentType). Similar principles are applied to Network data (datNetwork, datNeworkviz) and Affiliation data (datAffiliation, datAffiliationAuthor).
The structure of the database may seem redundant, for example, the author's biographical information (datAuthor and datAuthorName) could have been merged into one file, but the separation serves a   function. This splitting enables the SSHPA system to filter out overlapping author names faster because: (i) a Vietnamese author might have his or her name written differently in different publications, and (ii) the names recorded in our database are in Vietnamese spelling which has some digraphs and the addition of nine accent marks or diacritics.
As we now understand how the data are structured in the database of the SSHPA system, next we will examine how SSHPA can help improve control over the quality of data.

Data quality assurance and control
The basic principle for building a good data verification process here is to ensure four intertwined layers of check are always carried out: (i) inter-data-sources check: different publicly accessible sources were used to cross-validate the accuracy of collected data; (ii) inter-data-types check: the different types of data collected were checked for coherence with one another; (iii) inter-data-collectors check: the data collectors involved in this study cross-checked the information collected by each other, especially contents that have raised doubts over accuracy; (iv) random and periodic check. In each step, every mistake would be classified either as a one-off or systematic type and corrected accordingly.
In the SSHPA system, based on the above principle, the process is divided into quality assurance, which refers to the techniques implemented prior to entering data, and quality control, which indicates the techniques implemented after data is entered to check for errors. Another way the quality of data could be improved is to spot strange pattern in the data through generating network visualization of authors or articles' connections. The codes that are relevant to these processes can be found in Data Citation 1's Codes for SSHPA.pdf.

Quality assurance
The purpose of this step is to prevent bad data from ever being entered into the database in the first place. Several logic tests have been built into our semi-automatic system to help recognize suspicious authors or articles' data. For the authors' data, there are tests for:   -whether the name of an author already existed in the database -the name of an author must not be blank -if the author is Vietnamese, his/ her SSHPA ID must start with 'v'; 'f' if foreign author -if the author is female, her SSHPA ID must have the 'f' followed the initial 'v' or 'f'; 'm' if male author; '?' if sex is unknown -the correct format of SSHPA ID must be 'geography specifier + sex specifier + number'; for example: vm.1 is a Vietnamese male researcher numbered 1 or ff.1001 is a foreign female researcher numbered 1001.
For the articles' data, there are tests for: -whether the article of the same title already existed in the database -the title for the article must not be blank -the publisher and journal of the article must not be blank -the year of article publication must fall in the range 2008-now -fuzzy search article title for 90% similarity Failure to meet these requirements and the system will notify or even block the data collector from moving to the next data points in some cases. The data, when being entered, will also be changed to match the format designated by the system. For example, the paragraph break, the quotation mark (") ascii 147 code will be changed to (") ascii 34 in the title of the articles.

Quality control
This step is about applying the data validation tools to control the quality of data. The data validation tools include data filter, the search function (for relative and unique subjects), and the automatic data check functions. Here are some examples of these data validation tools.
Two authors with different SSHPA-IDs but same full names or middle names could easily be compared. And if they are suspected as being one person, the software can perform a three-step verification: -Through name: Check the author's name with all other authors with the same name in the system -Through affiliations: Check the author with all other authors with the same affiliation -Through publication: Check the author with all others with the same publication Furthermore, the software could filter out the low-quality data such as: -Authors with missing or invalid information: year of birth, sex, affiliation, article.
-Articles with no authors A notable feature of our quality control is that our data team members have invited the Vietnamese researchers to cooperate by directly verifying their information in our database. Though we have yet to hear from all of them, the responses we got to date do raise the credibility of the open database.

Automated construction of network data
There are several kinds of network data being automatically recorded with SSHPA: co-authorship among authors (undirected network data), leading-author to non-leading author(s) connection (directed network data), co-authorship among affiliations, co-authorship among geographical locations, etc. The network data allows for different ways to representing the data visually as shown in Figs. 5-8. This function enables the data collector to visualize the connections among the articles and authors in the database, thus providing him or her a new way to spot strange patterns in the data.

Code availability
The codes that are relevant for the data quality assurance, quality control and automated construction of network data of the SSHPA system could be found in (Computer Codes, Data Citation 1).

Data Records
The datasets are available from the Open Science Framework repository (Data Citation 1), under 'NVSS

2017-18 Scientific Productivity and Collaborative Networks of Vietnamese Researchers in the Social Sciences and Humanities.'
Below is the description of the main datasets that were produced by the SSHPA system, which can be found in Data Citation 1's Data files folder.   32 . Here, each dot represents a researcher that has a connection with Phan Van Phuc, a researcher with SSHPA-ID vm.780. Purple is coded for male, blue is coded for female; the square shape represents foreign researchers while the round shape is for the Vietnamese.

Technical Validation
Solving the problem of data duplication Thanks to cross-validating among various sources including research publication database Scopus, our SSHPA software found that of 34,629 articles indexed by Scopus open database, 463 articles have completely similar author names, which include (seemingly) Vietnamese full names and abbreviated names. The 5,414 authors associated with these articles turn out to have different Scopus EIDs. Many of those are of different nationalities such as Korean, Chinese, Taiwanese, etc. Moreover, given that any software would assign a unique ID to a unique name, any slightest variation in a name could yield a different ID. In this case, when looking at duplicated Scopus EIDs, we found that the most duplicated articles have up to five Scopus EIDs. Indeed, if scientific output researchers in any country set out to analyse the performances of individuals or institutions in their country, the example above shows just how difficult it is to only rely on query data directly from the Scopus open database. In our process to construct the SSHPA database, we found that the names of Vietnamese authors in the Scopus database are often not consistent, which poses a significant cost to data verification. Solving this problem requires a system that allows comparison of all name versions for any two authors. The SSHPA system has a built-in name-generating tool: once the full name of an author is entered into our database, the software will automatically generate all possible versions of names for a Vietnamese author. For example, an author with the full name "Nguyen Ngoc Anh" could generate 12 different versions of name: [0]: "Nguyen, Ngoc Anh" [1]: "Nguyen, N. A." [2]: "Nguyen, Anh Ngoc" [3]: "Nguyen, A. N." [4]: "Ngoc, Nguyen Anh" [5]: "Ngoc, N. A." [6]: "Anh, Nguyen Ngoc" [7]: "Anh, N. N." [8]: "Ngoc, Anh Nguyen" [9]: "Ngoc, A. N." [10]: "Anh, Ngoc Nguyen" [11]: "Anh, N. N." Searching in the Scopus database all these versions of this name gave us 32 results. As we filtered by author name and affiliation, the search result gave two Scopus profiles of a person named 'Nguyen Ngoc Anh' who worked at the Development and Policies Researcher Center, Hanoi, Vietnam. In our system, this mistake is eliminated and Mr Nguyen Ngoc Anh's profile is managed under one ID, the SSHPA-ID vm.1 (Data Citation 1). This allows our system to track any changes in the public profiles as well as update our own changes. This is one of many similar examples where the Scopus system creates two or more Scopus EIDs for one person. This is possible because one person can be affiliated with many organizations. In the SSHPA system, we could use the author validation tool to search for authors with similar names then using other data points such as affiliations, year of birth, fields of study, articles, etc. we could check these authors are the same person or not.

Diversifying datasets and data reports
The SSHPA software's ability to generate many different kinds of datasets and data reports sets it apart from the time-consuming and rigid manual system (Computer Codes, Data Citation 1). Not only does this function increase the data readability for users but it also helps the system admins detect any potential anomalies, thereby, able to improve the data quality as a whole.
The SSHPA software can generate four main kinds of report, as summarized in Table 3. Examples of the reports could be viewed in Data Citation 1's 'Extracted and Computed Data' folder, in which the number of articles according to fields, institutions, journals, publishers and years is shown.
As SSHPA is expected to correct for the delay in data update present in many citation indexing databases, the system will inform users of any authors or articles that were not yet found on such services. In the example of Mr Nguyen Ngoc Anh, thanks to cross-checking with other open resources such as Google Scholar, journal websites, institution websites, etc., we were able to update four of his papers that were published in 2016 and 2017, indexed in Google Scholars but were not yet found in Scopus as of 1 st February 2018 (Data Citation 1). Given that Vietnamese social scientists are expected to corroborate their profiles in the SSHPA database, this information will be both accurate and up-to-date.
Similarly, to improve data quality, the system's error report will list the missing information in the database, notifying the admins of any authors, articles, or affiliations that are duplicate or lacking details. The other two kinds of report give users a full picture of the system whenever necessary, one showing the statistical distribution of authors, articles according to gender, age, fields of study, etc., and the other providing the network data per specific requirements such as citation network or co-authorship network. An example of a general statistical report produced by SSHPA can be found in Table 4. Such reports are helpful in highlighting any abnormal data point, whether that be an incorrect count of authors, papers or affiliations.
For further details, please visit Data Citation 1's folder 'Extracted and Computed Data', in which details of the number of articles according to fields, institutions, journals, publishers and years are recorded.

Improving data visualization and research applicability
Just as the creation of data reports facilitates the validation of data quality, the SSHPA software's ability to generate descriptive statistics of networks and their visual maps also contributes to the overall quality of the database as well as its applicability in research on scientific productivity. The next part will go over the three main features and how they could help flag a mistake during data input.

Visualization of incorrect data entry
For the most part, given the rigorous data quality assurance and quality control, the maps of networks generated by the software are rarely prone to errors. However, in case of error, the system's ability to visualize selected scientific groups could flag the admins of any inconsistencies. For example, Fig. 5  case of both correct and incorrect network visualization of the data extracted from the article in 2017 by Phan et al. 32 . As the author Phan Van Phuc with SSHPA ID vm.780 is the first author, the correct network visualization should have the arrows go from him to his co-authors (fm.3086, fm.3084, fm.3085) as in Fig. 5b. When we incorrectly input the author fm.3084 as the first author, the network changes as shown Fig. 5a; the arrows now go from fm.3084 toward other co-authors. In this way, the network visualization tool helps the admins recognize when a mistake in the ordering of the authors has been made.

A map of Vietnamese social scientists' productivity
The datasets produced by SSHPA, such as the one in Table 5, could be used for cross-section and multivariate statistical analyses, allowing us to explore how networks of Vietnamese social scientists have grown over the years 17,18 . • Network data of scholars in one or many institutions (with or without direction, weight of non-weighted) Citation network • Network data of citation among people (with direction, weighted or non-weighted) • Network data of citation among articles (with direction, weighted or non-weighted)  The growth of scientific groups Another feature of the SSHPA database is allowing users to study how a group of researchers who have co-authored with each other evolves over time 17 . Figure 7 is an example. First, we chose a researcher with SSHPA ID vm.4 and set his level of connection to two (meaning the network data will be limited to researchers two walks away from vm.4). Then, we picked the time periods for comparison: from 2008 to 2010 (Fig. 7a), from 2008 to 2014 (Fig. 7b), and from 2008 to 2018 (Fig. 7c).

Network representation of all social scientists in the database
Besides enabling the visual representation of any scientific group within the database, the built-in function of network generator could also produce the visualization of all Vietnamese scientists and 973 foreign scholars (Data Citation 1) over different periods. Figure 8 is the result of this function. Figures 7 and 8 have different ways of representing the researchers' geography and sex. This flexibility provides the data collectors with diverse angles in examining the data patterns. In Fig. 7, colour represents sex while shape represents geography (Vietnamese or overseas); thus, both Vietnamese and foreign authors are represented equally. Figure 8 highlights the growth of research networks between Vietnamese authors and overseas, with three colours denoting the sex of Vietnamese scholars and the status of scholars who are foreign. In this way, network visualization enables a better reading of the data, thereby, improving the quality of the open database.

Usage Notes
As the SSHPA system relies only on open and free recourses to collect and verify data related to scientific output, it opens the possibility of replicating the same system in other places throughout the world (Computer Codes, Data Citation 1). It is also applicable for collecting and verifying data in other fields such as biomedicine, math, biology, etc. This work, therefore, aims to reduce the cost of doing science 10,33,34 .
The datasets of this study are first and foremost suitable for analyses of the relationship between individuals' scientific output and demographic (such as sex, age, region) as well as their collaboration characteristics. Moreover, with the development of the SSHPA software, one can also get access to citation and co-authorship network data, which would allow for more in-depth analyses of scientific influence among the researchers in the datasets.
We are convinced that the practice of open data is one of many ways to help cure the crisis of reproducibility in social sciences 10,34,35 , and improve policy-making processes when it comes to the question of funding efficiency 36 . We wish for scientists all over the world working on the issue of scientific productivity to replicate the design of the SSHPA system to verify the usefulness of the system in cleaning and eliminating data errors.   Table 5. Descriptive statistics of networks generated by SSHPA. Together with the numerical understanding of networks, SSHPA can also provide a visual representation of the collaboration patterns geographically, among Vietnamese provinces or between Vietnam and the world (See Fig. 6).