Technical and artistic production by brazilian postgraduate courses with emphasis on Social Sciences and Humanities

: Understanding production types helps define the impact of different knowledge areas. Information from the Sucupira postgraduate database was taken. Cluster and correspondence analyses were conducted to determine the behavior of different areas. Social Sciences (


Introduction
This paper aims to look at the production profiles from Brazilian postgraduate courses, especially in the Social Sciences (SS), Humanities (H), Letters, Literature & Arts (LLA), and examine for whom they are producing this knowledge. It looks deeper into the results from McManus and Baeta Neves (2021), identifying sources of funds and other differences between areas of knowledge. Overall, the abbreviation SSH will be used to contemplate the three areas together.
It is more complicated to define and collect information on research's social and political impacts when compared to bibliographic production and in the Social Sciences (SS), Humanities (H) and Letters, Literature & Arts (LLA) than in the hard sciences Baeta Neves, 2021). Most authors agree that assessing these impacts is complex (Bastow;Dunleavy;Tinkler, 2014). Authors argue that impact should be measured as the process rather than the outcome (Molas-Gallart; Tang, 2011). Differences exist between what is produced within each area of knowledge and why and how this information is produced (Ochsner;Hug;Daniel, 2013). Pedersen, Grønvad and Hvidtfeldt (2020) note the methodologically solid pluralism needed for analysing these areas, which affects the outputs. According to Prewitt, Schwandt and Straf (2012), the information produced in all science fields needs SS to explain whether, how and why that knowledge is used. Chen et al. (2015) stated that bibliographic methods could not fully capture the variety of products in SSH scholarship. Benneworth and Jonbloed (2010) found the valorisation of a universities' worth solely through patenting, licensing, spin-off formation and technology transfer as a worrying development, limiting the impact of Social Sciences and Humanities (SSH). This is partly because models for the hard sciences are generally not transferable to SSH. Universities' main stakeholders include the international scientific community, industry, politics, the public sector, and the general public (Jongbloed;Enders;Salerno, 2008), which demand a return from their investments. In this sense, it is more challenging to show the impact of SSH as it tends to be more time-dependent and sometimes less tangible than the (Science, Technology, Engineering and Mathematics) STEM areas. According to Benneworth and Jongbloed (2010), the universities do not promote the values of the SSH, especially in comparison to STEM areas. Nevertheless, the knowledge produced by these areas is used by government, business organizations, and cultural sector as well as information and technology industries (Olmos-Peñuela; Castro-Martínez; D'Este, 2014).
According to Kenyon (2014), implicit evaluations and comparative judgments without explicit criteria are fertile ground for inconsistency, arbitrariness and bias across a broad class of domains (e.g., Greenwald;Kreiger, 2006;Uhlmann;Cohen, 2005). Holbrook, Barr and Brown (2013) name 56 possible measures of research impact, with Esko, Tuunainen and Miettinen (2012) stating that quantitative indicators cannot capture the whole array of social and cultural impacts of research. Spaapen and Van Drooge (2011) proposed that impact in SSH should be measured through productive interactions, direct or personal; indirect through texts or artefacts; and financial through money or "in kind" contributions. Sovacool (2014) show that a broader pool of expertise is needed to understand how human behaviour affects demand and uptake of technologies, with SSH being marginalised. Pimenta (2016)  Felt (2014) also show that SSHs are portrayed as crucial for attaining the innovation goals yet are conceptualised as the junior partners; the leading role remains with science and engineering. Esteves, Franks and Vanclay (2012) highlight the need for culture, community, power, human rights, gender, justice, place, resilience and sustainable livelihoods as part of social impact assessment (SIA). They state that it is incumbent on SIA practitioners to educate proponents, regulators, and colleagues about these concepts and embed them into practice norms.

Literature review
The Brazilian postgraduate system was created to qualify human resources and produce scientific and technological knowledge to allow the country's industrial expansion (Baeta Neves, 2020;Moreira;Velho, 2008). In the context of the University Reform of 1968 (Silva, 1977;Durham, 2004), graduate studies serve to institutionalise research at Brazilian universities.
Debates about the emergence of new ways of producing knowledge and changing paradigms in Science and Technology (S&T) (Moreira;Velho, 2008) focus on the evaluation model. The main challenges of this system are changes in political, economic and social contexts, different from those in which postgraduate studies have been organised to date in Brazil. Moreira and Velho (2008) also point out that the acceleration of globalisation boosted the idea that complete, multi-institutional innovation systems should link science, technology, government and industry. In this context, human resource training has become even more critical to increase the country's competitiveness, training highly qualified researchers from universities and the graduate system to work in scientific production and new cuttingedge technologies. Thus, the concern with science and technology's environmental, social, economic, and political impacts must be part of human resource training for research in today's society (McManus; Baeta Neves, 2021). The principles that guide the organisation and evaluation of postgraduate courses (PG) in Brazil still retain typical features of a simplistic design. Supovitz (2009) showed that accountability responds to a real need in the educational system to demonstrate that public money is spent wisely. Highrisk testing encourages educators to align curriculum, standards and assessments.
When parameterising postgraduate products (which facilitates and speeds up the evaluation process) that do not adequately contemplate the different disciplinary profiles (McManus; Baeta Neves; Maranhão, 2020;McManus et al., 2021), academic institutionalisation would lead to a drop in the creation of relevant knowledge and a homogenised scientific production. As the quantity of output is pursued and articles published in scientific journals are prioritised, the result would be institutionalised disciplines that lead to partial, ultra-specialised and limited scope results. This would be detrimental to long-term research, as well as the construction of more complex theories and comprehensive explanations (Marenco, 2014), with wider scope and impact. The limitation of knowledge within pre-determined assessment areas remains the majority (Maranhão, 2010), and we need more researchers' openness to overcome the paradigm of disciplinary fragmentation (Schmitt et al., 2006). The Brazilian postgraduate system (Moreira;Velho, 2008) is based on principles of strictly academic recognition of the quality of courses, whose rewards are associated, among other criteria, with publications and the production of knowledge (Vogel;Kobashi, 2015) according to a linear pattern, from basic to applied science and, then, to development and production (Nobre;Freitas, 2017). This has caused a certain apparent homogeneity between the programs and an impatient search for concepts instead of accurate qualitative results. In addition (Bueno, 2015), under these conditions, showed that the Program's productivity is revealed as a quantitative production of publications in the channels evaluated as valid by the formulators of CAPES's evaluation standards, and efficiency becomes synonymous with the achievement of these standards. Therefore, the differentiated use of the resources for this production, notoriety or public effect of what they produce is not considered.
For Moita (2002), the ways currently used to evaluate the performance of postgraduate programs have been based on measurements conditioned to pre-fixed standards by the agencies and institutions that evaluate them. These hinder the potential efficiency existing in the group and the peculiarities they contemplate. According to Davyt and Velho (2000) and Baeta Neves and McManus (2020), what seems to have been maintained throughout this time is the notion that only scientists themselves can evaluate the work of their colleagues. Part of these objections is due to a lack of information on the parameters conventionally used in the academic evaluation process and responsible for institutionalising a discipline (Altman, 2012). Lopes et al. (2011) show that the internal collaboration of the graduate course in the country is a good indicator of the course grade.
It can be claimed, when prioritising quantitative indicators, such as the number of published articles, citations or impact factors, CAPES' institutional evaluation would end up encouraging the quantity of scientific production at the expense of its quality or originality (McManus; Baeta Neves, 2021). It may also promote subterfuge to artificially increase these rates, such as self-citation, cross-reference or replication of the same production (Mishra et al., 2018).
Shifts in research policy and reductions in government spending have meant that research, which was historically viewed as a social endeavour for the public good, is being recast in terms of productivity, economic efficiency, accountability, and value for money. Such shifts are part of a broader "audit culture" which has emerged in Higher Education Institutions (HEIs) (Shore;Wright, 2015). A new social contract has now arisen between science and state, which revolves around accountability, relevance and value (Demeritt, 2010). Therefore, research evaluation has become prominent in many industrialised countries (Guena; Martin, 2003). These research evaluation systems, which were once associated with assessing scientific output, are now increasingly associated with assessing societal impact as governments endeavour to ensure academic research is accountable and beneficial to society. With the emergence of societal impact assessments, a situation has emerged in which evaluators have no experience in applying this new, untested and unclear criterion (Derrick; Samuel, 2016;Samuel;Derrick, 2015 184 Van Leeuwen (2013) states that current bibliometric tools are not suitable to assess the actual research production of scholars in SSH. This author suggests that a broader focus on a wider variety of publication types is necessary, and these are not always intended for a purely academic audience. In this sense, much bibliographic production may have a technical use, or technical production published in bibliographic form (Aquino; Cortese; Shibao, 2019). Spaapen and Van Drooge (2011) show a need to narrow the gap between research and impact, focusing on productive yet reliable interactions. Accepted indicators are not available for social impact assessment. These authors recognise problems with the lack of quantitative data, lack of consensus on what data to collect, audience definition and expectations, attribution of impact and temporality. Olmos-Peñuela, Castro-Martínez and D'Este (2014) show that use of knowledge produced in universities in higher when there is interaction with external stakeholders.
According to Henshall (2011), research leads to a wide range of benefits or types of "payback".
According to these authors, much research leads, by subsequent application of the knowledge it generates, to significant benefits for society beyond that which the knowledge itself brings to science. It can be challenging to understand the ways and extent to which any individual project, researcher, or programme of research does this, even looking back with the wisdom of hindsight. It is even more challenging to predict, looking forward without the wisdom of hindsight, the characteristics of projects, researchers, or programmes that are likely to be associated with the greatest payback or impact. Volberda et al. (2010) show that 75% of successful innovation depends on social innovation, such as new forms of organising work and relations, and only 25% on R&D and new knowledge. In this case, the evaluation procedure should recognise that research, to achieve social impact, must cross disciplinary boundaries and those of other professional expertise and social domains.
New research assessment frameworks increasingly include previously untested criteria, considering how research is considered excellent or relevant outside the academic sphere (Derrick; Samuel, 2016). Donovan (2011) stated that metrics-only approaches employing economic data and science, technology and innovation indicators are in the past. Best practices combine relevant qualitative and quantitative indicators to gauge research's broader social, environmental, cultural and economic public value. The author states that it is often impossible to connect research and "impact" outcomes in a maze of complex social interactions and serendipitous turns. So, a cruder approach may be necessary (Martin, 2011). Molas-Gallart and Tang (2011) encourage "contribution not attribution" evaluations, while impact need not be conceived of purely in terms of economic returns but can embody broader public value in the form of social, cultural, environmental and economic benefits. Methods such as the Payback Framework (Donovan; Hanney, 2011) and SIAMPI (Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society) or "Participatory framework" (Rowe;Frewer, 2005) have the flexibility to accommodate these and many other impact domains. Muhonen, Benneworth and Olmos-Peñuela (2020) state that the interactions and the changes they mediate need to be monitored. They define 12 pathways in which SSH can impact societal change through dissemination, co-creation, reacting to societal change, and driving societal change.
Productive interactions are seen as exchanges between researchers and stakeholders whereby the Individual lecturers (as well as students, technicians etc.) maintain the Lattes database up-to-date, but only lecturer information is imported into Sucupira. Scholarships, courses/programs, lecturers, students and production in artistic, technical and bibliographic databases were downloaded from Sucupira for the period from 2013-2016 (last whole evaluation period). In theory, all activities by postgraduate lecturers in Brazil should be registered on the Lattes platform and therefore imported into the Sucupira platform. The platform uses data in three broad categories: bibliographic, artistic and technical. This paper looks mainly at the latter two, although a subdivision of books and chapters is highlighted insomuch as it refers to Technical services were divided into political and socio-economic as shown in Reale et al. (2018). The paper also looks at books and book chapters, which although bibliographical in nature may be of technical use. Data on reviews of papers or abstracts for journals, congresses, funding agencies etc., were not considered in this analysis as they have been published elsewhere (McManus; Baeta Neves, 2022).
The funder of the technical work was used to form word clouds. These consider the number of times the identification of who funded the research appears in the data bank. Subdivisions of technical and artistic (and therefore used in the construction of word clouds) are defined by the Sucupira platform.

Statistical analyses included correspondence analyses using SAS® (Statistical Analysis System
Institute, Cary, North Carolina). Correspondence analyses (PROC CORRESP) were carried out with all data and by area of knowledge and funder of the technical work and also separated if the funder was socio-economic or political.

Results
While exact and life sciences productions are concentrated in scientific journals and congresses, SSH tends to be more diversified, with a high output of technical services. This implies that the super valuation of one type of production over the rest may be prejudicial to these knowledge areas. Life and Exact sciences produce relatively more in congresses and journals than SS, H and LLA (Graphic 1). Annex 2 summarises different production types by area of knowledge, expanded from Graphic 1. Humanities, SSA and especially LLA, are most active in the visual arts (Table S2). The SSH areas tended to carry out more services than other areas (Graphic 3). This may be due to the importance of these services for the areas in question. It should be remembered that although this information was collected from the Sucupira database used to evaluate the postgraduate courses in Brazil, this information is also used by lecturers for career advancement or for the attainment of resources for research, among others. All areas highly interact with the business sector (Graphic 4). Agricultural sciences and engineering showed a higher percentage of work for businesses while SS, H and LLA more for governments (local, state or federal).
Medical and Applied SS have more international funding, as well as humanities (Graphic 3 and 4). This is probably because international organisations such as the World Bank, Unesco and other international organisations are of quality due to their international agenda and Brazilian production in this area.
Types of products such as Apps, Newspaper articles, patents, maps and translations differ between areas of knowledge (

Socio-economic
Funding or supporting production related to art and culture is varied (Figure 1). Applications for computer or cellphone and books and chapters showed funding from government agencies such as Capes and CNPq. Note the demand for maps made by SEEG (System for Estimates of Emissions and Removal of Greenhouse Gases) and WRI (World Resources Institute). Culture has the support of the Rouanet Law (Brasil, s./d.) (that supports cultural activities in Brazil) and SESC and numerous other entities, mediating funding for music and the arts. Other funding sources may be available but not registered in the Lattes database. The production of books and chapters (Graphic 7 and 8) in some cases may be considered technical production, such as manuals, professional works and didactic material. This definition was given by the lecturer when filling in his/her curriculum. While bibliographical in nature its use may be more technical. The production of books and chapters is similar by area of knowledge. While most areas produce these documents from their research projects, Human, SS, LLA and multidisciplinary also produce much didactic and technical material, reinforcing the need for production in Portuguese, with 86,9% in Portuguese, 11% in other languages (mainly English) and 2% in more than one language. There was little difference between languages for the different types of production, varying from 79% in Portuguese for translations to 90,6% for Professional works.

Discussion
Archambault and Larivière (2011) showed three limitations of bibliometrics in the analysis of the impact of SSH. They show that assessing the impact of the humanities applied social sciences and linguistics, letters and arts, as previously stated, must also include the socio-economic and political dimensions. Reale et al. (2018) stated that political impact refers to the transfer of research findings to the political sphere to inform decision-making or policy design. On the other hand, social impact refers to how an action from a policy or a civil society-led action has contributed to improving identified social challenges.
Thereby, by identifying the studies' funders in the present paper, we can see who the potential users of this information are and the possible impact. According to Kenyon (2014), SSH products include books, According to Reale et al. (2018), impact is often understood as a change that research outcomes produce upon academic activities, the economy, and society at large. Social Sciences and Humanities show that international, national and local governments and public and private organisations use SSH products (Graphics 5 and 6; Figure 1). Identification of funders can also help to identify possible end-users.
While the financing of book and chapter production by CAPES and CNPq is in line with these agencies being the main funders of post-graduate education in Brazil, they also were main funders for computer apps, as by-products of research projects. Other productions show financing from both governmental and According to Benneworth and Jongbloed (2009), universities have generally become a representative for the "arts" community in governmental discussions. According to these authors, the strategy has become essential in positioning universities as important cultural actors. While there might be a lot of rhetoric, these authors state that valorisation comes through engagement with community stakeholders. Figure 1 shows the funders for scenic and visual arts in Brazil count on support from museums, theatres, and federal and local governments. Book reviews may be used. Still, data on Brazilian books in these databases are scarce, maybe because of the language barrier or because they are not used as citable references but rather as didactic or historical sources or for the general public (Sivertsen;Larsen, 2012). This use cannot be seen as less important than citable scientific papers as it may cater to cultural peculiarities of students, encourages learning through activities contextualized in the environment, and contribute to the preservation of local heritage (Seoane;Rodríguez, 2015). Use of this type of material also varies by subject area (Cooke; Rosenthal, 2011;Roesnita;Zainab, 2005). As such, SSH tends to have a more context-sensitive and nuanced understanding of impact (Pedersen;Grønvad;Hvidtfeldt, 2020), including book production for general audiences. McManus et al. (2022) show that bibliographic production in SSH have wider impact than citation or bibliographic analyses, especially those analysed using bases such as Web of Science and Scopus. SSH also has many more research and technical reports, manuals and didactic material than other areas (Graphics 1, 7 and 8). Neylon, Willmers and King (2014) also showed that these types of bibliographic production are not always cited, but also what is cited (such as scientific papers in journals) may not be used.
It is possible to determine what an area of knowledge produces in Brazil. This would mean modifying the Curriculum Lattes in Brazil to collect this information more efficiently (McManus; Baeta Neves, 2021). Keeping this curriculum up-to-date is an annual obligation for all lecturers and students in postgraduate programs. It is also used to request grants and scholarships and for career advancement.
Therefore, it is a powerful tool for measuring the political and socio-economic impact of research in Brazil but needs to be modified for this end.
Pedersen, Grønvad and Hvidtfeldt (2020)  It should be noted that a change is underway in Brazil, with pressure being put on these areas to move away from the traditional manner of production (technical production, books, chapters, congresses) and into the publication of scientific papers, especially in international journals, as happens in other areas of knowledge. As discussed above, this is not a simple task. The evaluation of SS, H and LLA needs to be better understood and examined so that the relevant products from these areas is not lost. Nevertheless, quality and relevance must be maintained when publishing scientific papers (McManus; Baeta Neves; Maranhão, 2020).

Conclusion
This paper aims to look at the production profiles from Brazilian postgraduate courses, especially The funding sources for these products also vary by product and subarea of knowledge.

Conflict of interest
The authors declare no conflict of interest