What’s being tested and what’s being learnt? A contribution to lessons learned evaluation methods for community-based sustainability initiatives.
DOI:
https://doi.org/10.29015/cerem.596Słowa kluczowe:
lessons learned, evaluation, community development, sustainability policy, project managementAbstrakt
Aim:
There is little good practice guidance with respect to methods and skills for conducting lessons learned evaluations of community-based development projects. In this paper we utilise a mixed methods approach to evaluate the lessons learned by the team members and stakeholders of a funded five year ‘test-and-learn’ sustainability initiative. The approach combines a statistical and a qualitative thematic analysis of transcribed textual data and presents an analytic framework with which to track the lessons learned by community development projects.
Design/ Research methods:
A mixed methods approach combining text and sentiment mining complemented by a qualitative thematic analysis is applied to the same data collected from stakeholder responses to an on-line survey and the transcribed audio recordings of four focus groups in which stakeholders participated.
Conclusions/ findings:
Employing replicable tools, augmented by qualitative research methods, provide a framework for a systematic approach to elicit and capture lessons learned by a sustainable community development project. These bear on how project activities, from engagement to supporting the local food economy, have been experienced by stakeholders and their learning acquired over the course of the project. Implications for future project design and funding options are considered.
Originality/ value of the article:
Despite the evident value of its contribution to improving project design and funding options, the evaluation of lessons learned in community-based sustainability work remains under-researched. The combined use of text and sentiment mining techniques with qualitative thematic analysis on the same data offers an original contribution to research in this field.
Bibliografia
Anbari, F. T., Carayannis, E. G. and Voetsch, R. J. (2008) ‘Post-project reviews as a key project management competence’, Technovation, 28(10), pp. 633–643. doi: 10.1016/j.technovation.2007.12.001.
Argyris, C. and Schön, D. (1992) ‘What is an organization that it may learn?’, On organizational learning, pp. 8–29.
Bartsch, V., Ebers, M. and Maurer, I. (2013) ‘Learning in project-based organizations: The role of project teams’ social capital for overcoming barriers to learning’, International Journal of Project Management, 31(2), pp. 239–251. doi: 10.1016/j.ijproman.2012.06.009.
Big Lottery Fund (2012) £12m lifeline for vulnerable facing fuel and food poverty, Big Lottery Fund: News and Events. Available at: https://www.biglotteryfund.org.uk/global-content/press-releases/england/archive/12m-lifeline-for-vulnerable-facing-fuel-and-food-poverty (Accessed: 3 April 2016).
Boisot, M. H. (1999) Knowledge assets: Securing competitive advantage in the information economy. New York: Oxford University Press.
Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, 3(2), pp. 77–101. doi: 10.1191/1478088706qp063oa.
Bruner, J. (1986) Actual minds, possible worlds. Cambridge, Mass.: Harvard University Press.
Bryman, A. (2012) Social research methods. 4th edn. New York: Oxford University Press.
Carrillo, P., Ruikar, K. and Fuller, P. (2013) ‘When will we learn? Improving lessons learned practice in construction’, International Journal of Project Management, 31(4), pp. 567–578. doi: 10.1016/j.ijproman.2012.10.005.
Davenport, T. H. and Prusak, L. (1998) Working knowledge: How organizations manage what they know. Boston, MA: Harvard Business School Press.
Duffield, S. M. and Whitty, S. J. (2016) ‘Application of the Systemic Lessons Learned Knowledge model for Organisational Learning through Projects’, International Journal of Project Management, 34(7), pp. 1280–1293. doi: 10.1016/j.ijproman.2016.07.001.
Feinerer, I. and Hornik, K. (2014) ‘tm: Text Mining Package.’ Available at: http://cran.r-project.org/package=tm.
Feinerer, I., Hornik, K. and Meyer, D. (2008) ‘Text Mining Infrastructure in R’, Journal Of Statistical Software, 25(5), pp. 1–54. Available at: http://www.jstatsoft.org/v25/i05.
Grimmer, J. and Stewart, B. M. (2013) ‘Text as Data : The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts’, Political Analysis, pp. 1–31. doi: 10.1093/pan/mps028.
Guba, E. G. and Lincoln, Y. S. (1989) Fourth generation evaluation. Newbury Park, CA: Sage Publications.
Hahn, U. and Mani, I. (2000) ‘The challenges of automatic summarization’, Computer, November, pp. 29–36.
Hillard, D., Purpura, S. and Wilkerson, J. (2007) ‘Computer-Assisted Topic Classification for Mixed-Methods Social Science Research’, Journal of Information Technology & Politics, 4(4), pp. 31–46. doi: 10.1080/19331680801975367.
Jockers, M. (2017) ‘Package “syuzhet”: Extracts Sentiment and Sentiment-Derived Plot Arcs from Text’. Available at: https://cran.r-project.org/web/packages/syuzhet/syuzhet.pdf.
Kayser, V. and Blind, K. (2017) ‘Extending the knowledge base of foresight: The contribution of text mining’, Technological Forecasting & Social Change, 116, pp. 208–215. doi: 10.1016/j.techfore.2016.10.017.
Kumar, B. S. and Ravi, V. (2016) ‘A survey of the applications of text mining in financial domain’, Knowledge-Based Systems, 114, pp. 128–147. doi: 10.1016/j.knosys.2016.10.003.
Laver, M., Benoit, K. and Garry, J. (2003) ‘Extracting Policy Positions from Political Texts Using Words as Data’, The American Political Science Review, 97(2), pp. 311–331.
Levitt, B. and March, J. G. (1988) ‘Organizational Learning’, Annual Review of Sociology, 14, pp. 319–340. doi: Doi 10.1146/Annurev.So.14.080188.001535.
Lindner, F. and Wald, A. (2011) ‘Success factors of knowledge management in temporary organizations’, International Journal of Project Management, 29(7), pp. 877–888. doi: 10.1016/j.ijproman.2010.09.003.
Love, P. E. D., Teo, P., Davidson, M., Cumming, S. and Morrison, J. (2016) ‘Building absorptive capacity in an alliance: Process improvement through lessons learned’, International Journal of Project Management, 34(7), pp. 1123–1137. doi: 10.1016/j.ijproman.2016.05.010.
Makrakis, V. and Kostoulas-Makrakis, N. (2016) ‘Bridging the qualitative – quantitative divide: Experiences from conducting a mixed methods evaluation in the RUCAS programme’, Evaluation and Program Planning, 54, pp. 144–151. doi: 10.1016/j.evalprogplan.2015.07.008.
Maurer, I., Bartsch, V. and Ebers, M. (2011) ‘The Value of Intra-organizational Social Capital: How it Fosters Knowledge Transfer, Innovation Performance, and Growth’, Organization Studies, 32(2), pp. 157–185. doi: 10.1177/0170840610394301.
McClory, S., Read, M. and Labib, A. (2017) ‘Conceptualising the lessons-learned process in project management: Towards a triple-loop learning framework’, International Journal of Project Management, 35(7), pp. 1322–1335. doi: 10.1016/j.ijproman.2017.05.006.
Meaney, C., Moineddin, R., Voruganti, T., Ann, M., Brien, O., Krueger, P. and Sullivan, F. (2016) ‘Text mining describes the use of statistical and epidemiological methods in published medical research’, Journal of Clinical Epidemiology, 74, pp. 124–132. doi: 10.1016/j.jclinepi.2015.10.020.
Mohammad, S. M. and Turney, P. D. (2013) ‘Crowdsourcing a Word–Emotion Association Lexicon’, Computational Intelligence, 29(3), pp. 436–465.
Moustaghfir, K. and Schiuma, G. (2013) ‘Knowledge, learning, and innovation: research and perspectives’, Journal of Knowledge Management, 17(4), pp. 495–510. doi: 10.1108/JKM-04-2013-0141.
Newton, J., Franklin, A., Middleton, J. and Marsden, T. (2012) ‘(Re-)negotiating access: The politics of researching skills and knowledge for “sustainable communities”’, Geoforum, 43(3), pp. 585–594. doi: 10.1016/j.geoforum.2011.12.003.
Nonaka, I. and Takeuchi, H. (1995) The knowledge-creating company: How Japanese companies create the dynamics of innovation. New York: Oxford University Press.
Office of Government Commerce [OGC] (2009) Managing Successful Projects with PRINCE2. London: TSO (The Stationery Office).
Patton, M. Q. (1997) Utilization-focused evaluation. 3rd edn. Thousand Oaks, CA: Sage Publications.
Powell, J. H. and Bradford, J. P. (2000) ‘Targeting intelligence gathering in a dynamic competitive environment’, International Journal of Information Management, 20(3), pp. 181–195. doi: 10.1016/S0268-4012(00)00004-9.
Putnam, R. D. (2000) Bowling alone: The collapse and revival of American community. New York: Simon and Schuster.
R Core Team (2014) ‘R: A language and environment for statistical computing.’ Vienna, Austria: Foundation for Statistical Computing. Available at: http://www.r-project.org/.
Rennie, D. L. (2012) ‘Qualitative research as methodical hermeneutics.’, Psychological Methods, 17(3), pp. 385–398. doi: 10.1037/a0029250.
Rittel, H. W. J. and Webber, M. M. (1973) ‘Dilemmas in a General Theory of Planning’, Policy Sciences, 4(2), pp. 155–169. doi: 10.1007/BF01405730.
Rolstadås, A., Tommelein, I., Morten Schiefloe, P. and Ballard, G. (2014) ‘Understanding project success through analysis of project management approach’, International Journal of Managing Projects in Business, 7, pp. 638–660. doi: 10.1108/IJMPB-09-2013-0048.
Saif, H., He, Y. and Alani, H. (2012) ‘Alleviating data sparsity for twitter sentiment analysis’, CEUR Workshop Proceedings, 838, pp. 2–9.
Schein, E. H. (1996) ‘Three Cultures of Management: The Key to Organizational Learning’, Sloan Management Review, 38(1), p. 9.
Sutcliffe, K. M. and Weber, K. (2003) ‘The High Cost of Accurate Knowledge’, Harvard Business Review, 81(5), pp. 74–82. doi: 10.1109/EMR.2003.24900.
Thomas, W. H. (2015) The basics of project evaluation and lessons learned. 2nd edn. Boca Raton, FL: CRC Press.
Ur-Rahman, N. and Harding, J. A. (2012) ‘Expert Systems with Applications Textual data mining for industrial knowledge management and text classification: A business oriented approach’, Expert Systems With Applications, 39(5), pp. 4729–4739. doi: 10.1016/j.eswa.2011.09.124.
Williams, T. (2003) ‘Learning from projects’, The Journal of the Operational Research Society, 54, pp. 443–451. doi: 10.1057/palgrave.jors.2601549.
Pobrania
Opublikowane
Numer
Dział
Licencja
Autor przenosi nieodpłatnie na Wyższą Szkołę Bankową we Wrocławiu , bez ograniczeń terytorialnych, majątkowe prawa autorskie do tego utworu w rozumieniu ustawy z dnia 4 lutego 1994 roku o prawie autorskim i prawach pokrewnych ( Dz.U. 1994, Nr 24, poz. 83 ze zm. )na zasadzie wyłączności, tj. prawo do:
a) wyłącznego używania i wykorzystania utworu w dowolnej działalności przez Wyższą Szkołę Bankową we Wrocławiu, w szczególności w działalność Biblioteki Cyfrowej uruchomionej przez Wyższą Szkołę Bankową we Wrocławiu
b) wytwarzania, utrwalania i zwielokrotniania egzemplarzy utworów wszelkimi technikami, w tym techniką drukarską, reprograficzną, zapisu magnetycznego oraz techniką cyfrową, w szczególności ich zwielokrotniania poprzez dokonywanie zapisów na płytach typu CD,
c) zamieszczenia wybranych fragmentów utworu w celach promocyjnych w publikacjach, materiałach promocyjnych, w sieci Internet oraz sieciach wewnętrznych typu Intranet Wyższej Szkoły Bankowej we Wrocławiu,
d) wprowadzania utworu do pamięci komputera Wyższej Szkoły Bankowej we Wrocławiu,
e) kopiowania i powielania utworu w technologiach fotomechanicznych lub innych znanych w dniu zawarcia umowy (fotokopie, kserokopie itp.),
f) przetworzenia dzieła na formę elektroniczną i nieograniczonego rozpowszechniania w sieci Internet.
