Читать книгу Позитивные изменения. Том 3, № 4 (2023). Positive changes. Volume 3, Issue 4(2023) - Редакция журнала «Позитивные изменения» - Страница 6

One of the pivotal events of this autumn for Russian professionals engaged in the field of evaluation is the annual conference hosted by the Association of Specialists in Program and Policy Evaluation. The central theme of the 2023 conference was assessing the quality of life and well-being. In this piece, we explore the experiences and practices shared by conference participants regarding the implementation and application of monitoring and evaluation in the social sphere.

Yulia Vyatkina

Editor, Positive Changes Journal

SHIFTING THE SPOTLIGHT TO HAPPINESS

The IX conference of the Russian Association of Specialists in Program and Policy Evaluation (ASPPE) took place online on October 17–18, 2023. This year’s conference featured 42 presentations and hosted 14 thematic and discussion sessions. The event drew over 500 attendees from all corners of Russia, spanning from Yuzhno-Sakhalinsk in the Far East to Kaliningrad in the West and Makhachkala in the South, the organizers say.

The conference program encompassed two main tracks: “Wellbeing-Focused Evaluation” and “Evaluation of Sociocultural Projects,” in addition to thematic segments dedicated to evaluation in specific domains – such as social entrepreneurship, working with children with special developmental needs, individuals with disabilities, and various forms of chronic illnesses. As usual, several sessions were also devoted to evaluation methodologies.

“The program was exceptionally rich, packed with engaging and substantive presentations. The relevance of the issues on the table and the profound discussions surrounding them underscore that interest in evaluation remains unwavering. I believe that thanks to our speakers and partners, whom I’d like to extend my heartfelt appreciation to, the conference offered an opportunity for all participants to expand their knowledge, enhance their professional competencies, delve into existing practices, and introduce their real-world working experiences to colleagues. In my view, this event holds great significance and utility, nurturing a sense of unity within the evaluation community and inspiring fresh achievements,” remarked Anna Lygina, President ASPPE.

The Accounts Chamber of the Russian Federation, which is responsible for evaluating the implementation of state programs and policies, among other things, has been an active participant in the conference for several years. In recent times, this regulatory body has been actively championing an evidence-based approach.

“In 2021–2022, we organized an extensive competition for government and municipal employees to encourage the adoption of an evidencebased approach in decision-making.[17] In 2023, in collaboration with the Financial University, we are hosting a research project competition for undergraduate and graduate students.[18] We aspire to cultivate a community of individuals passionate about evaluation among students, researchers, and decision-makers,” stated Anastasia Kim, Deputy Head of Inspection at the Research and Methodology Department of the Accounts Chamber of the Russian Federation, during the ASPPE conference’s opening session.

The central theme of this year’s conference revolved around the evaluation of quality of life and well-being. As early as the 1960s, researchers began to conclude that gross domestic product (GDP) and the standard of living were not the ideal indicators to describe society’s life and individual well-being. That is how the concept of “quality of life” emerged, oriented toward a comprehensive assessment of various aspects of societal life, including non-material ones.[19]

In 1972, the concept of Gross National Happiness was introduced by the King of Bhutan in place of the commonly used GDP. This immediately elevated a small South Asian nation to become a trailblazer among countries making happiness the cornerstone of national policy. In 2008, the kingdom developed the corresponding tool – the Gross National Happiness Index. Happiness indices have since been calculated for various countries. One example is the World Happiness Report, published by the UN’s Sustainable Development Solutions Network.[20] According to the 2023 report, Russia ranks 70th out of 137 countries in terms of happiness, with Finland, Denmark, and Iceland leading the chart.

THROUGH THE PRISM OF HAPPINESS AND WELL-BEING: AN OVERVIEW OF INTERNATIONAL EXPERIENCE

Approaches to assessing the quality of life, well-being, and happiness are intricately interlinked, as noted by Natalia Kosheleva, a member of the ASPPE Board and a consultant specializing in monitoring and evaluating socially oriented programs and projects.

According to the expert, these approaches can be categorized into three major groups, distinguished by their impact scale:

1. National/Territorial

Programs and policies aiming to enhance the quality of life, well-being, and happiness of people within a specific country or territory.

2. Organization-Specific

Programs or projects whose objective is to promote the quality of life, well-being, and happiness of individuals within a given organization.

3. Individual

The goal of programs and policies is to ensure the quality of life, well-being, and happiness of specific individuals.

The national/territorial approach can be illustrated by Bhutan, where the Gross National Happiness Index encompasses 9 spheres of human life:

• Psychological well-being;

• Health;

• Time utilization (e.g., adequate sleep is considered an important part of well-being);

• Education;

• Cultural diversity and sustainability;

• Quality management;

• Quality of life in local communities;

• Ecological diversity;

• Standard of living.

The survey questionnaire for the Gross National Happiness Index comprises 249 questions. For instance, in the Psychological Well-Being section, residents of the kingdom are asked to identify the 6–7 most important factors and elements that contribute to their happiness, rate the quality of their life on a scale of 0 to 10, and express how much pleasure they derive from it.

Which units can we measure well-being in? The Happiness Research Institute suggests a unit of measurement known as WALY (Wellbeing Adjusted Life Years) – life years adjusted based on the subjective level of well-being. This approach originates from the field of medicine, where health is measured in QALYs (Quality Adjusted Life Years) and DALYs (Disability Adjusted Life Years).

In essence, WALY is akin to a kilowatt-hour: if we have an electrical device with a power of one kilowatt, it consumes one kilowatt-hour of electricity in an hour of continuous operation. When calculating WALY, subjective well-being ratings on a scale of 0 to 10 are used as “power,” where 10 signifies complete life satisfaction. If a person is completely satisfied with their life for a year, the well-being “consumed” amounts to one WALY, Natalia Kosheleva explains.

Researchers also strive to calculate well-being losses and gains resulting from various factors and policies. Moreover, WALY enables the measurement of the Happiness Return on Investment (HROI).

According to Natalia Kosheleva, there is a significant development in the assessment of how projects impact individuals’ subjective well-being. For example, substantial work in this direction has been carried out in the United Kingdom. There is a center dedicated to evidence-based approaches in projects aimed at enhancing well-being. Since 2011, the annual population survey features four questions related to personal well-being. Respondents are asked to rate their current life satisfaction on a scale of 1 to 10, assess the significance of their life pursuits, reflect on their happiness the previous day, and indicate their level of anxiety during the same period.[21]

Approaches to assessing the quality of life, well-being, and happiness are interlinked. They can be categorized into three groups, based by their impact scale.

In the UK, creative approaches to measuring well-being have also emerged. Convinced that the purpose of exhibitions includes influencing people’s emotional and psychological states, museums have particularly excelled in this area. To assess this impact, all visitors are invited to select a leaf of a specific color at the entrance to the exhibition and attach it to a tree. Following a color-coded system, a red leaf signifies “I feel good and prosperous,” a yellow leaf indicates a neutral state, and a green leaf expresses “I feel sad.” At the exit, visitors encounter another tree. Subsequently, the number of leaves of each color at the entrance and exit is tallied to gauge whether the exhibition has affected people’s emotional states.

RUSSIAN EXPERIENCE

Over the past three years, various indices for measuring happiness and well-being have also made their debut in Russia. The pioneer was the Russian Cities Quality of Life Index, developed in 2021 by VEB.RF in collaboration with PwC (now known as Trust Technologies) and the Russian Academy of National Economy and Public Administration, in partnership with the Agency for Strategic Initiatives (ASI). This comprises an information and analytical system that, as of 2023, consists of several components:[22]

• A database encompassing data on over 300 indicators for 218 cities across Russia. It allows analyzing and comparing cities, including with their foreign counterparts (based on individual indicators).

• City profiles, offering a detailed portrait of each city and facilitating comparisons of specific indicators with the national average or a cluster of similar cities.

The Timchenko Foundation also developed the General Index of Child Well-being, introduced in 2021, followed by the RUSAL Cities Quality of Life Index in 2022. These indices share a similar structure and rely on specific indicators. In the following sections, we will delve into each of these indices in more detail and explore how they can be utilized to assess conditions in various regions.

RUSAL’S CITIES QUALITY OF LIFE AND SUSTAINABLE DEVELOPMENT INDEX

This index, introduced in 2022, forms the basis of the company’s new social investment strategy. Irina Bakhtina, RUSAL’s Chief Sustainability Officer, explained that it is founded on criteria used by the Organization for Economic Cooperation and Development (OECD) as well as the VEB.RF Quality of Life Index.[23] The key distinction between RUSAL’s index and VEB.RF’s database lies in the population size of the territories under examination. RUSAL is interested in evaluating the quality of life and sustainability in cities and communities with populations of below 50,000.

66 indicators were considered in the development of this index, including statistical data (32 indicators), survey data (19 indicators), and information obtained through requests to regional and local authorities (15 indicators). RUSAL’s comprehensive tool covers 21 territories, including cities and municipal districts, across 12 dimensions of quality of life, categorized into three aspects: Values, Environment, and Potential.

“Values” include the following characteristics: 1) Work-life balance, leisure opportunities, and consumption of goods and services.

2) The ability to maintain regular contact with family and acquaintances, trust in the community, and a willingness to participate in social activities.

3) Satisfaction with life and confidence in the future.

4) Public safety.

“Environment” includes:

1) A comfortable, appealing, and diverse urban environment, encompassing streets, parks, and public spaces.

2) Good health and access to advanced medical care.

3) Favorable environmental conditions and an advanced waste management system within the city.

4) Availability of high-quality, well-maintained, and conveniently located housing.

The Happiness Research Institute suggests a unit of measurement known as WALY – life years adjusted based on the subjective level of well-being.

“Potential” comprises the following characteristics:

1) Access to employment opportunities that enable income and professional growth.

2) Extensive opportunities for receiving quality education for both children and adults.

3) Convenient transportation within the city and the ability to travel beyond its borders.

4) Effective governance and protection of civil rights.

“We consistently emphasize: this is not a ranking; it’s a diagnostic tool. It is crucial for us to identify areas where we may be falling short and to understand why. It is essential to examine the results in a comparative context. Why is this important? If our employees and their families move from one plant to another, we want to assess how their quality of life might change,” explains Irina Bakhtina.

Based on the results of 2022, regional centers such as Volgograd, Krasnoyarsk, Sayanogorsk, Novokuznetsk, along with the Sharypovsky district of the Krasnoyarsk Krai and Kandalaksha in the Murmansk region, took the lead in the index rankings. Conversely, the territories of the Irkutsk region lagged behind. For instance, Taishet, a city in the Irkutsk region with a population of 34,000, occupied the 21st position out of 21 in terms of socio-economic well-being. Respondents provided a low subjective rating of urban improvement quality, highlighted inadequate street lighting, noted low participation of children in extracurricular activities, reported low scores in the Unified State Exam (USE), mentioned a scarcity of events and cultural institutions in the city, pointed out a high accident rate, and cited limited affordability of housing.

Understanding the landscape of these issues serves as a crucial reference point for making decisions regarding social investments. In this regard, the company hopes that by addressing significant problems in Taishet and other cities and regions, it will witness improvements reflected in the index in the years to come.

CHILD WELL-BEING INDEX

The Child Well-Being Index serves as a data source to gauge the quality of life for children. Such indices are employed in numerous countries across the world. For instance, there is a report from Ireland’s Department of Children’s Affairs on the state of the child sector in the country, the Child Poverty and Hardship Index in the Kyrgyz Republic, the National Cohort Study and School Health Surveys in Finland, and Kazakhstan’s Child Wellbeing Index.

In Russia, the Child Well-Being Index[24] was developed with contributions by the Timchenko Foundation, the Russian Academy of National Economy and Public Administration, Lomonosov Moscow State University, Moscow State University of Psychology and Education, National Research University Higher School of Economics, the Public Chamber of the Russian Federation, and the Russian Ministry of Education.

This instrument consists of two indices: the Statistical Index of Child Well-Being, calculated based on 27 indicators published by Rosstat, and the Subjective Index of Child Well-Being, derived from 18 indicators obtained through surveys of children and their parents.

The Statistical Index encompasses four domains: health, development, material well-being, and security. Meanwhile, the Subjective Index includes six domains: health, material well-being, safety, education, social relationships, and self-actualization. The data collection system for the Subjective Index is designed to allow measurement not only at a regional level but also at the level of specific schools. Within each domain, responses to three questions are analyzed. For example, in the “Material Well-Being” domain, these questions pertain to assessing the family’s financial situation, whether the child has personal belongings and whether the child has been on vacation.

The creators of the index emphasize the importance of separately measuring the statistical and subjective indices of child well-being, with special attention given to situations where children’s assessments fall below the statistical figures.

“Of course, for us, the priority is not just the numbers that compile the ranking. As a foundation, our primary concern is the well-being of the child. In other words, how the data from such a comprehensive study influences management decision-making based on an evidence-based approach,” explains Elina Illarionova, Head of the Analytical Service at the Timchenko Foundation.

It is important to separate measuring the statistical and subjective index of child well-being, with attention given to situations where children’s assessments fall below the statistical figures.

According to the Russian Child Well-Being Index for 2022, families with children are facing financial challenges, with only 50 % of surveyed children rating their family’s financial situation positively. Additionally, there is room for improvement in children’s access to leisure activities, as 55 % of surveyed children had not traveled anywhere in the past year. Children’s self-esteem is relatively low, with only 45 % of them reporting achievements and successes they take pride in. Furthermore, there is potential to enhance children’s motivation and interest in learning, as 70 % of the surveyed children expressed liking school, and 65 % found their lessons interesting.

These findings can serve as the foundation for practical initiatives. For example, in Altai Krai, day care groups were established for children from families facing financial difficulties, allowing parents the freedom to work or pursue further education, thereby increasing the family’s income. In Nizhnevartovsk, libraries have created spaces for children and teenagers to explore their interests. In Langepas, summer camps and evening leisure activities were organized, enabling children to discover new talents and interests. In the Tambov region, efforts are being made to boost children’s interest in learning through the expansion of Olympiad programs and the introduction of new supplementary education programs in various fields: arts, natural science, technology, tourism and local lore, social science and pedagogy, physical education and sports.

According to Elina Illarionova, the developers of the index collaborate with regional analytical groups, often based at universities, which can provide practical recommendations to specialized agencies. Both the Subjective and Statistical Indices of Child Well-Being are publicly accessible and can benefit a wide range of stakeholders, including regional and federal authorities, NGOs, social investors, the expert and scientific community, as well as parents and children.

“We strive to ensure that the index data is integrated into regulatory frameworks to lay the groundwork for child welfare programs. On top of that, we believe that presenting the research results should involve the active participation of children, and that the data should not only inform management decisions but also justify the need for child-centric policies,” emphasizes Elina Illarionova.

WHAT POSITIVE PSYCHOLOGY CAN OFFER AS A FIELD OF SCIENTIFIC KNOWLEDGE

How does an individual’s perception of happiness and well-being relate to themselves? Are external circumstances in which a person lives influencing this perception? Why is it essential to incorporate subjective and psychological well-being into our evaluations? Answers to these questions can be found by delving into the realm of positive psychology.

When we mention “positive psychology,” most often, it conjures associations with affirmations, a positive mindset, and optimistic thinking. In other words, “think positively, and everything will be fine.” However, this is not the complete picture. Positive psychology, as a distinct branch of psychological knowledge and practice, explores the resources within individuals, their satisfaction, values, talents, and their connection to happiness. These discussions were also part of the Wellbeing-Focused Evaluation track.

The concept of subjective well-being emerged in psychology during the 1970s when the age-old philosophical question of happiness became a focal point for empirical investigation within this field, as explained by Anna Hegai, a graduate student and teacher at the Higher School of Economics, and a psychologist-methodologist at the Victoria Charitable Children’s Foundation.

Researchers identify three approaches to measuring well-being:

1. Subjective well-being (E. Diener) – how closely a person believes their life aligns with the most desirable state.

2. Psychological well-being (C. Ryff, D. Leontiev) – the assessment of personality traits that contribute to happiness (positive relationships with others, self-acceptance, having a life purpose, etc.).

3. Quality of life – an objective evaluation of the favorability of external living conditions.

As far back as 1970, while studying well-being in Nordic countries, Finnish sociologist E. Allardt categorized all human needs into three groups: having, loving, and being.[25] For each category, both objective and subjective measures of satisfaction were examined. Only individuals themselves can determine how complete, fulfilling, and meaningful their lives are – in other words, their subjectively perceived quality of life.

Renowned researchers, including Nobel laureate Daniel Kahneman and his collaborator Amos Tversky, have discovered that perceived quality of life and subjective well-being are minimally affected by external circumstances. They found that “life circumstances make only a small contribution to the variance of happiness – far smaller than the contribution of inherited temperament or personality.”[26] This phenomenon has been termed the paradox of subjective well-being.

The encouraging news is that if you possess a sufficient level of optimism and life satisfaction, this state of well-being can be quickly regained, regardless of external events. Anna Hegai explains, “After significant life shocks, such as the loss of loved ones, divorce, or severe illness, individuals typically regain their previous level of well-being within 3–5 years. Additionally, people tend to become more content and good-natured with age. This reflects a form of personal growth associated with aging. Individuals in their 40s tend to report greater life satisfaction than those in their 17s.”

According to a model proposed by UCR professor Sonja Lyubomirsky,[27] happiness can be attributed to the following factors:

• Genetic predisposition (50 %);

• Life circumstances (10 %);

• Intentional activities and behaviors (40 %).

While genetic predisposition and life circumstances cannot be easily influenced, behavioral change is within an individual’s control. People can alter their outlook on life, thought patterns, habits, and self-esteem.

Moreover, engaging in activities that demand intentional effort, such as continuous learning, plays a pivotal role in achieving lasting happiness. This implies that external circumstances, which are not related to intentional activity, have a fleeting impact on subjective well-being,[28] whereas purposeful activities involving effortful challenges and volitional choices form the foundation for sustained well-being.

Psychological science also provides diagnostic tools for measuring personal resources, perceived support, life orientation, and subjective well-being. Anna Hegai lists several well-being indicators that can be assessed before and after the implementation of social and psychological projects and programs:

1) Resilience (the ability to recover and grow after facing stress).

2) Endurance (engagement, risk-taking, control).

3) Sense of connection (understanding, manageability, meaningfulness in life).

4) Optimism.

5) Self-esteem.

6) Self-efficacy (belief in one’s ability to cope).

7) Tolerance of uncertainty.

8) Self-control.

As Hegai notes, incorporating relevant questionnaires (e.g., resilience assessments, basic needs satisfaction scales, anxiety and depression screenings, and future perception questionnaires) into the evaluation of social projects and programs yields valuable insights into their impact.

CONCLUSION

As demonstrated by ASPPE conference 2023, approaches to assessing well-being and happiness are diverse and highly intriguing. Experts emphasize the importance of using subjective and objective methods in tandem, as they harmoniously complement each other. Evaluating the effectiveness of each of these approaches requires the development of its unique set of tools. Furthermore, conference participants consistently highlighted how various analytical instruments employed in both approaches facilitate data-driven decision-making.

Turning our attention to other sessions at the ASPPE conference, the sessions within the Evaluation of Sociocultural Projects track garnered significant interest, with more than 150 attendees, according to the organizers. The presentations within this track revolved around the experiences of both large foundations that support cultural projects, necessitating the evaluation of their program outcomes, and small NGOs engaged in project assessment.

In the upcoming issues of Positive Changes Journal, we will dive deeply into some of the themes explored during the conference presentations. You can access recordings of all conference sessions on the ASPPE VKontakte page[29] and the Association’s YouTube channel.[30]