Tips & Tricks for Child Focus Groups, Part 1

Written by CRC on . Posted in

by Mandi Singleton


(Note: this post is the first part of a two-part series.)


Elementary classroom. Focus on teacher standing in front of chalkboard.

First tip — don’t expect things to be this orderly!

One of my favorite things about my job is conducting focus groups. I enjoy the opportunity it gives me to interact with people, capturing and learning from their thoughts and feelings about experiences they’ve had. While at CRC I’ve had the opportunity to facilitate a series of focus groups with elementary school students.


Although many of my projects are education-related, I had never done a group with children so young before. The focus groups I’d done in the past involved middle grade students, parents, and school staff, so the thought of conducting focus groups with elementary school students made me a little nervous.


I could just imagine rambunctious 6 to 10 year olds, hopped up on sugar and far too excited to break away from their schools’ typical routines and reigns of control to participate in a focus group. I guess my main concerns in conducting focus groups with such young children were getting them involved, keeping them engaged, and capturing genuine but thoughtful responses.


Because school-aged children are still developing (physically, socially, emotionally, cognitively), the way they think, communicate, and interact with others differs from adults. These developmental differences point to the importance of identifying focus group strategies that are specifically catered to children’s communication competencies, as techniques used in focus groups with adults would not be effective. My overall goals for focus groups with young children are to ensure that the participants understand my questions, have the opportunity to reflect on their own experiences, and as a result can effectively communicate their thoughts and feelings.


Thankfully, I’ve found that by using the right strategies that my young focus group participants’ excitement eventually succumbed to attentiveness as the group format played to their inquisitive natures.


scsckids Here are some of the tips and tricks I’ve found to work for focus groups with children:


1. Be mindful of group composition. To increase involvement, levels of engagement, and quality of responses, research suggests limiting groups to four to six participants that are no more than two years apart in age or level of development. In my experience, I’ve been able to limit each session to six children; for example, one group was conducted with 1st and 2nd graders, while another only included 3rd and 4th graders. I found that limiting participants was beneficial in fostering engagement, while controlling for large age-discrepancies seemed to help prevent students’ responses from being overly influenced by their peers.


2. Build a trusting atmosphere and relaxed setting. Children are more likely to be engaged by focus groups that foster relaxed settings where they feel comfortable enough to express their thoughts and feelings. To facilitate this type of setting, research suggests that moderators use ice-breaker games, engage in casual (but age appropriate) conversation with participants before the start of the session, portray a friendly and relaxed manner, and encourage the use of first names. In my focus groups with children, participants were invited to do an ice-breaker activity at the beginning of the session, which did build trust between participants and helped them to relax. The students paired up with a partner in order to learn something about each other, and took turns introducing their partner to the rest of the group. The result was a relatively quiet group of children, more comfortable with each other, who then became more talkative in an appropriate way as the session progressed. Fostering a certain atmosphere when doing groups in schools is especially important; I’ve found it most effective for children to view me, as the moderator, in a more informal way than they do their teachers to encourage their honest responses.


3. Establish ground rules. Research suggests establishing ground rules at the start of each focus group, as they help children understand their role in the group, what is expected from them, and what they can expect from the moderator. At the beginning of each session, I’ve asked participants to abide by basic discussion rules (e.g. be respectful, be good listeners) and informed them why I wanted to talk with them. I let them know anything they said in the group would not be shared with anyone else with their names attached, and that they didn’t have to respond to any questions they didn’t want to. Before starting the focus group, children were also given the opportunity to ask any questions they had. Note that challenges have arisen for me in soliciting honest responses; this occurred when children observed peers and wanted to model and/or conform to peers’ responses. However, I was able to resolve these situations by varying my methods (more about this in Part 2).


4. Consider your Interview structure and question formation. Research on focus groups supports that groups with school-aged children should start off with simple questions that can be answered with brief one word responses (e.g., yes or no) and progress to more complex or multipart questions. This eases children into the interview process, making them more comfortable with responding to the moderator. The full focus group guide should primarily consist of open-ended questions, with direct questions only used as a means to clarify or elicit more detail on a response. Close attention should be paid to the wording of questions to ensure age appropriateness and that students understand what they are being asked. In groups I’ve facilitated, children were read statements that probed for feelings about their social lives and interests in math, reading, and science. The focus groups started with a few warm-up questions that asked about their feelings towards vanilla ice cream and rainy days; not only were these questions helpful in getting the children comfortable with the interview process, but they also reassured me as the moderator that the children understood how to correctly respond using tools I provided to support non-verbal responses to augment verbal ones (more on this, also, in Part 2). Responses elicited during a drawing activity, for example, were followed-up with more direct questions in an effort to stimulate additional discussion and gain further insight.


I hope that the above tips give you some food for thought and a starting point for your data collection with this unique population. Stay tuned for Part 2, including how to engage your groups with FUN activities, coming later this month!

Visual Reports

Written by Taj Carson on . Posted in , Dataviz, Technology and Customer Service

By: Sheila

Several weeks ago, one of our clients came to us with a challenge: find compelling ways to present 10 years of grantmaking data. The client wanted us to tell their story and present the data in a way that people at all levels (data nerds and non-data nerds) at their organization could easily understand.

I was tasked with analyzing the data and worked closely with the CRC dataviz experts, Taj and Matthew, to come up with the different visuals for the report. I’m no dataviz expert but here’s what I learned:
  • Client feedback is important: Take time to hear your client’s thoughts on the visualizations you are creating, you want to make sure you are meeting your client’s expectations.
  • Patience is key: I spent a lot of time creating and re-creating multiple charts and graphs. It takes time to make sure every visual aligns with the story you are trying to tell. If you need a break, eat a muffin.
  • Don’t be afraid to ask for help: If you get stuck, ask a colleague or check out online forums to see if there is a solution to the problem you’re having. Google is a great friend.
  • Sketch!: Take time to sketch out what you want your visuals to look like. Trust me, it’ll save you a lot of time in the end.
  • Don’t be afraid to try: I made about 40-50 visuals for this project. Around half were rejected by the design team but I learned a lot throughout the process:
    • Pie charts are not your friend
    • No one at CRC likes pink or mustard yellow
    • Embrace white space
    • Not every visualization needs to be a bar graph
    • Embrace awesomeness

In the end, we created a pretty cool report for our client that they really liked. Since we can’t share the report online for proprietary purposes, we created a similar report for this blog. Take a look and share your thoughts.

Also, don’t forget to check out our summer webinar series:

Data Systems: Where and How to Store Your Data

July 15, 2015 :12-1pm

Sarah will explore different options for data storage systems and solutions for those who seek to streamline their data collection and storage processes. This learning session will focus on selecting the best software for your specific needs and organizing your program data.

Microsoft Excel Magic: Developing Mesmerizing Charts that Enchant & Engage your Audience

July 29, 2015: 12-1pm

This webinar will focus on the effective visual representation of data using charts that can be created in Microsoft Excel. The strategic use of design principles and the practical skills needed to create charts that are visually appealing, functional, and informative will be addressed

Each webinar costs $25. Sign up here:

And this great map of outdoor film locations in Baltimore created by Matthew:

Happy Summer!


What Breastfeeding in the U.S. Looks Like

Written by CRC on . Posted in

CRC’s dataviz team recently completed a comprehensive and beautiful infographic documenting breastfeeding statistics in the United States. Our hope is that this infographic can play a part in spreading the word about this important issue.


Screen Shot 2015-05-29 at 10.47.55 AM


From a public health standpoint, the medical benefits of breastfeeding are well established.* Breast milk provides babies with all the necessary fats, proteins, and vitamins they need for healthy growth and development. Among other benefits, antibodies in breast milk can help babies fight infections and reduce the risk developing asthma and allergies. Moms who breastfeed experience benefits too. Breastfeeding can help mom lose her pregnancy weight (through the calories it burns) and can protect her against breast and ovarian cancers. In addition to the physical benefits, time spent breastfeeding also helps nurture the bond between mom and baby.


See the full infographic here:!breastfeeding-in-the-usa/ccrt


* Although the above benefits of breastfeeding are established in current research, we would like to acknowledge that other bodies of research demonstrate that babies who are not breastfed also can have healthy outcomes and bond well with moms. Many moms cannot or may not choose to breastfeed for a wide variety of reasons, and we support all moms in their choices.

We LOVE Maps: Map out your summer!

Written by CRC on . Posted in

As we’ve said in the past, we at CRC LOVE MAPS. They’re useful and (often) beautiful, helping us to make all kinds of decisions in research and daily life.


Some of you have been following our interest in maps at the Baltimore DataMind blog, but to make sure more of our readers get to see that content, starting with this post we’re “folding” the BDM blog in here. So now, along with the evaluation news, data tricks, and dataviz tips you’ve come to expect from CRC’s blog, expect to learn more about making and using maps. We also hope to show you a lot of just plain cool ones, starting with this one, created by our own Matthew Earls.


Baltimoreans’ love of festivals is possibly even greater than our love of maps! This map uses an interactive and chronological format to map out all the festivals in-town this summer. Use it to map out your summer plans!



A Word About Baltimore City’s Snappy Budget Graphic….

Written by CRC on . Posted in

By: Taj
Infographics are all the rage. They are beautiful, engaging, and fun to look at. This one is no exception:    
Available for download here:

Available for download here:

At first glance, it looks like a lot of fun. The Finance Office has done something unexpected, which is trying to make understanding the budget of Baltimore City a bit easier by using data visualization. It is likely to be successful in that more people will look at this than might read a website that breaks out funding by categories, or talks about the property rate; but, it also leaves a lot to be desired.

Here are a few things we would suggest to the City of Baltimore for next time:

Pie charts
Be careful with them. Unlike other data visualization folks, I’m not completely opposed to the pie chart in all situations. But, if you are going to use them, you should have the slices go from the largest (property taxes at 32%) to the smallest (other at 4%) slices. This one does work okay because you don’t have a million slices.

The colors seem randomly chosen (except for the green tree and blue water) and the color palette on the top half of the infographic looks unrelated to the bottom half. Pick a few colors and stick with them throughout so the piece looks consistent. Consider gradient colors (i.e., a color fill that gradually blends from one color to another) for the Priority Outcomes.

The bottom graphic is hard to read. It might be the case that the color of the Priority Outcome is supposed to match the color of the pie slice, but it’s not clear. Are the colors communicating information or if is this just the color palette? If the former, you made a good choice as a way to unify the piece, at least the bottom of the piece. However, a bit of labelling of the bottom pie chart (like the top one) would have cleared this up.

And… What’s with the arrows? What are they pointing at?

We applaud the Finance Department for using data visualization to make the budget more accessible to citizens. And we love the clean look and feel of this piece. We’d love to see them use some of the best practices in the field of data visualization to make it even more impactful!

As a final note, thank you for NOT putting the City seal on this.

Why the Hell was Taj at a Design Conference and What Did She Learn There?

Written by CRC on . Posted in , CRC Team, Dataviz, Design

By: Taj
i15banner     Last week I was at the Interaction Design conference. Now, you probably know that evaluation and design don’t exactly go hand-in-hand, so I understand if your next thought is a befuddled, “Huh?” You don’t usually find evaluators at a design conference.     1-2So, was I just hopping a plane to San Francisco in February because of the awesome weather and the cool town? Well, not exactly, although I have to say those were both nice perks of being there. Did I travel to San Fran for some much needed R&R and to clear my head? Not entirely, but I do always feel inspired and innovative when I’m there. (And it’s a bit surreal to go by the Uber headquarters in your Uber.)             The truth is, I’ve been doing a ton of reading and thinking about design lately. The more I learn, the more I find that there is a common ground between the way we do our work here at CRC and the work of design firms. I’ll be writing a series of blogs in the future about how we use design thinking in our approach to research and evaluation, but for now I’ll just quote Phi Hong Ha, who said in her conference talk, “Design is about helping people to make sense of the world.”   That’s what we evaluators do, too! Help make sense of the world. It’s our job to help people understand what is going on, what their clients are thinking and feeling, and what the reality is of the communities they work in and what the impact is of their programs.     Design thinking, combined with CRC’s love of beautiful data, makes a design conference the perfect place for me to learn new things, embrace old ways of doing things with a new language, and continue to be inspired. As Tim Brown (Tim Brown!) said, “Information is the material we are most using in design today.”   TimBrown     If that doesn’t sound relevant to what we do here at CRC, I don’t know what does.     So, what did I get out of this conference?     dansaffer_1291256779_0Creative innovation is risky. Smaller firms and individuals are often able to innovate, try new things, and “play” more than larger firms because we smaller firms have a higher tolerance for risk and less bureaucracy to fight.One barrier you encounter to creativity, as Dan Saffer pointed out, is that “Efficiency is the enemy of creativity.” Creativity takes time. And it’s not always linear. Researchers love linear—linear relationships, linear regression. Here at CRC we have a certain tolerance for a bit of meandering if it means a better process, a better relationship, and a better outcome.     Things never turn out like you think they will. So stop expecting them to. It’s been our experience that projects never go exactly as intended. That’s because our work involves people, not widgets (or pharmaceuticals). And people change, make decisions, and adapt constantly. Jan Chipchase said, “Once you begin, assume that everything you planned is not relevant.” While this is an extreme perspective, I have found it very helpful to let go of the idea that nothing will change. Being flexible in our thinking has worked for us very well, and it’s a constant in the design world.     Silicon Valley is full of people telling you failure is great. “Fail fast” they say. Foundations especially struggle with this. They often hold onto approaches or initiatives long after they should let go and move on to new and innovative ideas. Everyone struggles with change, individuals and organizations. But Saffer made a good point. “Failure sucks!” he said. Learning from failure is where it’s at. We are constantly trying out new approaches and new tools. Some of them succeed, some of them fail. But I have yet to try out something new that we didn’t learn from, whether it was software, process, or research methods.     Risk     Another theme of design work is empathy. The idea of empathy often makes researchers uncomfortable. We embrace empathy here at CRC. Watching people work with their clients or fill out forms or struggle with databases gives us empathy for how we might develop solutions to make their work easier. Indi Young did a great job of defining empathy in a way that I think even evaluators and researchers can embrace. She differentiated between “emotional empathy” (feeling what someone is feeling) and “cognitive empathy”, which is about understanding how people are thinking and why and how they react.     I would argue that we need some of both because we also need to understand the emotional state people are in when we are asking them to fill out forms, take attendance, complete surveys, etc. Sometimes they are upset, frustrated, or anxious (forms can bring it out in the best of us). But cognitive empathy applies too. How do people see and understand our data collection tools? How are they interpreting the questions we ask on surveys (hello, cognitive interviewing)? What makes them skip over some parts of the forms they fill out? Observing and talking with people about these things specifically can be very fruitful.     dani2Danielle Malik wins the award for best presentation title with “Go Home Data, You’re Drunk.” She talks about how future trends will be all about analytics and customization (even more so than now). Because of the visual storytelling we do here, I was excited to hear “data points are the words, and it’s up to us to construct the sentences.” Her presentation resonated with me, as she focused on the fact that data is not neutral. You have to constantly think critically about where it comes from, why you collect it, and how you intend to use it. And we’ll definitely be looking for ways to use the hashtag from her presentation … #drunkdata.         I think evaluators have much to learn from the design community. Design is user centered, process oriented, and collaborative. The design process requires that you empathize with your users, understand what their problems are, come up with creative solutions, test them, and then build in a process of iteration and tweaking until you get where you need to be.     I think evaluation should be more like design. I will admit that I often have doubts about the way we design and implement evaluations in this field. Evaluators often do not take the users into account. Some evaluators work in virtual isolation from their end-users. How many actually spend time in the schools, public health clinics, and program sites where their data collection takes place (besides us, of course)? How many actually talk to the people filling out the surveys to find out what they think was meant? Karl Fass, professor of User Interface design said, “We shouldn’t be using the vocabulary of natural science,” and I agree. I often question why we use the vocabulary AND the methods of the natural sciences in a contest that is very human-centered and (let’s face it) often chaotic.     So I leave you with these questions….     What would happen if we used human-centered design principles and user testing to develop and evaluate social programs instead of evidence-based practices? [1]     What if we began every program implementation with empathy, planned it collaboratively with the end user in mind, and assumed that some iteration would take place before it got to where it needed to be?     What if we conducted our evaluations not like science experiments but like user analytics?     What if we were careful to collect only what we (or our clients) needed, to constantly review the data and collaboratively make decisions with it, and to not assume there was a beginning or an end point? How revolutionary would that be?     Well, if I’m going to be truthful, we know that good evaluation practices do these things, even though we don’t always like to talk about it. But in my upcoming posts on design thinking, I plan to do just that.   SteveJobs     [1] It’s a radical notion, but it’s one worth considering. Large, elaborate studies are conducted using the principals of the natural sciences. Then we plop those programs into a variety of locations as if context doesn’t matter. Designers know that context always matters. And that users can be almost infinitely segmented.

Secrets from the Data Cave: January 2015

Written by CRC on . Posted in Technology and Customer Service

By: Sarah


 Welcome to CRC’s monthly series of articles on all things techie: Secrets from the Data Cave! (For those who don’t know, the title references our room — fondly referred to as “the bat cave”— where data staff can geek out in an isolated setting.) Here we’ll be offering you a sneak peek into the cave, with tips and the latest updates on what we’re implementing here at CRC.

(This month’s SDC is an on-the-road edition!)




Late last fall, I had the opportunity to go to phpworld 2014, a conference for PHP code developers, held in Washington DC.


Many different web applications use the programming language PHP, so coders have a myriad of options for deploying it (including, but not limited to, Drupal, WordPress, Joomla, and Magneto). This conference is designed to bring these communities together in order for developers to learn about how others are using this language, and get ideas for enhancing existing applications.


The conference sessions that fascinated me the most were those about web security. PHP is great for developers because it is a very powerful language, meaning you can do a lot with it. But, without the proper precautions, hackers can exploit that power over your applications and cause huge headaches.




One conference presenter gave a great example of how this might look in the real world. As context for the example, you need to know that there are certain combinations of coding language that, should a hacker type them into your application and you haven’t protected your database, the application will interpret them as legitimate SQL code and “drop” all the tables in your database. Therefore a hacker could use this technique (called “SQL injection”) with a “drop database table” command – feeding the application some code that masquerades as part of the original developer’s code— to irreversibly erase all of your data.



With this in mind, consider the presenter’s example: A would-be hacker in Europe rigged up a fake license plate with a bit of SQL injection code on it. He was exploiting the fact that traffic cameras use picture-parsing technology to break down an image of a license plate into individual characters (to record the plate info of speeders).




So, this was presumably an effort to trick the traffic camera into parsing the code, feeding the data in as it would any license plate. This would have ultimately caused all the stored traffic data on the back-end to be wiped clean via the SQL injection! Now, it’s unlikely that this actually worked in practice. But, it did get me thinking about the lengths hackers will go to in order to mess with your data.


  All in all, it was an excellent conference, and I feel I learned a lot about this powerful coding language. I’m now looking forward to connecting with more PHP developers in future!

Data Visualization Predictions for 2015

Written by Sheila Matano on . Posted in

By: Taj and Sarah
Technology is always changing and evolving, and data visualization technology is no exception. While the term “infographic” was once unfamiliar to most people, fun visualizations are now regularly spotted on Facebook and Twitter feeds for both individuals and businesses.

But just how much can we expect to see change within a year? In the spirit of the New Year, we are posting a few predictions on where we think data viz is headed in 2015 and beyond:


Microsoft Office: it can do a lot more than you think

With all the integrated apps and changes in Office 2013, you can really accomplish some amazing visualizations using nothing more than your trusty old Microsoft Office software. But the fact is that a lot of people don’t use it to its full potential, and don’t realize what it can do.

We predict that in 2015, more people will figure out that Office can be used to create some really interesting visuals, like the chart of weather patterns below (an Excel template):   weather or this dashboard we created:



No more giant reports sitting untouched in file cabinets

Or, at least, fewer of them: in 2015 we predict a small but perceptible decline in the popularity of the traditional final paper report for program evaluation (you know, that one you meant to read, but never actually got around to, and then you lost it in your desk drawer?) With the rising popularity of infographics, dashboards, and even graphic memos to communicate findings, people will begin to gravitate towards these options for reporting program outcomes, as they are sometimes a better fit.


Less is more

Data viz gurus like Edward Tufte have been touting this advice for a long time, but it is our hope and prediction that more people this year will come to see that less is more when it comes to visualizing data using charts and graphs. Fewer colors, borders, and dancing bears can, in fact, make your data viz better (see this concept come to life at   Have a prediction you’d like to share? Leave it in the comments below!    

CRC’s 2014 In Review

Written by CRC on . Posted in Business operations, CRC Team, Dataviz

Warm Winter Tidings from all of us  at CRC!

Warm Winter Tidings from all of us at CRC!



We’ve had an extraordinarily busy year, and are so pleased to have shared much of it with you!  


This month, rather than a blog with evaluation-related info and tips, we’re once again ending our work year together by taking looks back and forward.    


CRC-Logo newAlong with continuing to provide high quality evaluation and data services to clients — both existing and new for 2014 — Carson Research has completed its long-term project of updating our branding and communications so that they better reflect what we do and who we serve. Heading into 2015, we hope you’ll become used to hearing from us as simply “CRC”, in addition to calling us that to save breath and/or typing time.    



To quickly catch you up on our 2014, our resident data viz department, held down by Ashley, beautifully summarized our year’s activities:


CRC Holiday Card Inside Front Cover    


But, as is our custom, we also asked all the CRCers to say a little (note: our definitions of “little” vary) something about their year. Below, our staff reflects on things they’ve experienced and learned from in 2014 and things they’re looking forward to in 2015:


  • Ashley, Database Analyst
This year I learned that infographics and dashboards are AWESOME! Utilizing fun icons, cool graphs, and eye-catching colors to tell data stories for our clients has been so much fun. I am looking forward to exploring more resources for creating them!
  • Dana, Research Assistant
I assisted with the implementation of a community survey and, having never done something like that before, I was able to learn a lot of new skills. Planning, implementing and analyzing the results of the surveys within a 2-3 month span was hectic to say the least, but the experience helped me gain a lot of valuable tools and resources. I also took a qualitative analysis course at The Evaluator’s institute in DC and an AEA online webinar on survey methods and sampling, which helped me learn new tips and tricks that I can incorporate into my current and future work. I also had the opportunity to learn some Arc GIS mapping skills and self-taught myself some rudimentary VBA coding skills on excel as I worked on a proposal database for CRC. I’m looking forward to working on new projects and learning new things from them, as well as utilizing some of the skills I have learned over the course of this year, such as mapping.  


CRC's annual holiday lunch for staff. This year at The Food Market (

CRC’s annual holiday lunch for staff. This year at The Food Market (

  • Jill, Research Analyst
In 2014 I was greatly relieved and proud to officially complete my PhD program at UMBC! My favorite learning opportunity of the year was attending Museum Camp at Santa Cruz Museum of Art and History. It provided me with inspiring and invigorating ideas about how to make evaluation more user-friendly, engaging, and even fun for everyone involved! Next year I hope to apply more of what I learned at camp, attend other, similarly enlightening conferences, as well as to continue honing my visualization skills.
  • Kevin, Controller
I tore my ACL in July throwing discus and had to drop out of track & field at the 2014 Gay Games. I postponed surgery and went on to win 3 gold and a bronze in my main sport of swimming at the Games. I am now rehabbing after cadaveric ACL replacement & meniscus surgery. I still have many months of PT ahead of me but have registered for the 2015 EuroGames in Stockholm, Sweden to be held in August. It’s on!
  • Leslie, Research Analyst
This fall, I began a second master’s degree in Educational Psychology. It’s been challenging fitting studying into my schedule and the online format of the program is a new experience for me, but I enjoy being back in school. I’m looking forward to the classes I’ll take next semester in educational measurement and assessment development.
  • Mandi, Research Assistant
What I learned this year: (1) Giving presentations isn’t so scary after all – even at national conferences. (This year I attended AEA 2014 as a first time presenter); (2) Focus groups with can be both fun and informative. (I performed a number of child focus groups this past year and found there were certain activities that kept kids more engaged during the session than others. Stay tuned for an upcoming CRC blog for details!). What I’m looking forward to next year: (1) More focus groups (I’m looking forward to conducting more focus groups next year, refining my interviewing skills, and finding different focus group activities that are both fun and engaging.); (2) Data visualization (I plan to incorporate more data visualization into the work I do. Including qualitative data visualization tools and techniques; (3) Attending conferences (I enjoy the workshops, networking with other evaluators, and traveling, especially to places I’ve never been to before).  


Opening our "Secret Krampus" gifts!

Opening our “Secret Krampus” gifts!

  • Matthew, GIS Analyst
This year I began learning how to code and to use that code to make more dynamic and customized maps. Next year I’m looking forward to expanding GIS skills and coding to grow the mapping services division here at CRC.
  • Sarah, Technology & Information Specialist
The coolest thing that I learned this year was how to program in PHP. It’s an incredibly powerful programming language! And next year, I am looking forward to launching our own online database application, using PHP and some other fun things!
  • Sheila, Research Analyst
The coolest thing I did this year was travel. At EERS, it was great to see old friends and hear Dr. Rog’s talk on infusing evaluation theory with practice. In New Orleans, hearing six surgeons general speak at APHA about their challenges, accomplishments and vision for the future was inspirational. In Detroit, I got to learn how Shinola makes their watches; it was also great to visit D3 and see how the organization works to provide communities with to high-quality data. In Minneapolis, I learned about different data viz techniques at EYEO and visited art installations at Northern Spark. In New York, I attended the launch of Data & Society and met great folks who are researching the social, cultural, and ethical issues arising from data-centric tech development. At the CIC impact summit in DC, I heard how various tools can be used to tell more meaningful, accurate, and connected stories that can improve community outcomes. All were great experiences, but watching world renowned rugby players compete  for the World Series Title at Sam Boyd Stadium was, by far, the most awesome thing I did this year. What’s next? In 2015 I’m looking forward to perfecting my chocolate mug cake recipe, learning more about data viz and improving my mapping skills. I’d also like to tackle learning a new language on duolingo.
  • Taj, CEO
The big game changer for me was attending the EYEO Festival in Minneapolis in June. EYEO brings together data geeks, coders, designers and artists for some of the most fascinating presentations and conversations I have ever experienced at a conference. It is THE data visualization gathering, in my opinion. I was so inspired by my experiences there, that I chose to take the big leap to go back to school. I’m now back in grad school, earning (another) Master’s Degree, this time in Information Visualization at MICA. Becoming a beginner again is very humbling, and I’m excited to see how what I learn in this program can help CRC go so much farther in our efforts to help our clients to visualize and understand their data.
  • Tracy, Research Associate
My biggest learning experience in 2014 was designing and implementing a community survey in two Baltimore City neighborhoods. In the coming year, I anticipate opportunities to work with new clients and expand my evaluation skill set.    


Krampus aftermath.

Krampus aftermath.



Happy New Year, and here’s to new learning, growth, and gainful opportunities… as well as more beautiful data… for you and yours in 2015!    


AEA 2014 Recap

Written by CRC on . Posted in , CRC Team

by Mandolin Singleton visionary_eval


Last month I attended the 2014 American Evaluation Association conference in Denver, CO. 2014’s conference theme was “Visionary Evaluation for a Sustainable, Equitable Future.” The event brought together research and evaluation professionals from all over the globe and from a variety of disciplines (e.g., community psychology, health and human services, PreK-12 educational evaluation). Attendees were encouraged to explore ways in which evaluation could be used to support sustainability and equality across disciplines and sectors.




This year’s conference was especially exciting (as well as nerve-wrecking) for me because I was attending as a first time conference presenter. I went to numerous sessions, learned a lot, and had a great time connecting with other evaluators. (I even found a little bit of time to explore Denver’s spectacular shopping scene). Below are some of my highlights from the conference.


    Robert Kahle: Dominators, Cynics, and Wallflowers: Practical Strategies for Moderating Meaningful wallflowersFocus Groups
Robert Kahle is a sociologist and expert in qualitative research. He is versed in leading skill building sessions for both new and experienced focus group moderators. In this workshop, he talked about how to effectively manage focus group dynamics, identified problem behaviors typically observed in groups, and reviewed strategies to recognize, prevent, and address them. I found this workshop especially informative and will be applying some of these techniques in my upcoming focus groups. If you’re interested in learning more about what was reviewed in this session, much of the content can be found in Robert’s book, “Dominators, Cynics, and Wallflowers: Practical Strategies for Moderating Meaningful Focus Groups”.





    Veena Pankaj: Data Placemats: A DataViz Technique to Improve Stakeholder Understanding of Evaluation Results
Veena Pankaj has experience directing evaluation design and implementation with a focus on participatory approaches (you can check out one of her recent publications on participatory analysis here). Pankaj described a data visualization technique (Data Placemats) she uses to engage, improve understanding, and solicit stakeholder interpretation of evaluation results. She talked about the logistics of the technique (e.g., the what, when, and how) and reviewed the learning journey involved in their creation. If you’re interested in Veena’s work, you can find slides and resources from the session on SlideShare.


My Presentations


Deciding to one-up myself by giving not only one, but two presentations my first go around didn’t really help my nerves, but what can I say, I was enthralled by this year’s conference theme (you could probably also say there was a little part of me trying to impress the boss as well). I gave a poster presentation on an infographic that we (CRC) created for Elev8 Baltimore in effort to visually display evaluation findings. The poster reviewed the process, results, and implications of translating data findings into a reader-friendly infographic.


Poster session reception

Poster session reception


My poster implied that infographics can be successfully translated into attractive, functional, and informative infographics. It suggested the use of infographics to report evaluation findings, as effective data visualization can attract readers, aid in interpretation of data, and support comprehension. Perhaps most importantly, it implied infographics can be used to promote the use of evaluation findings to inform decision making!


Me and my poster!

Me and my poster!


I also gave a paper presentation on our (CRC’s) establishment of an early warning system at the Elev8 Baltimore sites, and reviewed the application of early warning indicators to the middle grades. Within the presentation I gave a brief description of the system and reviewed the steps we took to create it – including everything from gaining access to the data to producing the final reports. In this presentation I described how early warning indicators can enhance the accessibility and use of evaluation data; through providing reports to sites on a quarterly basis, the early warning indicators can be used on a real-time basis to inform programmatic decisions. Implications were also made for expanding the use of early warning indicators from high school to middle school populations.


Between the multitude of great sessions I attended, learning loads of information, giving two presentations, and still finding time to scope out the shopping scene in Denver, you could say I had a whirlwind of a time at AEA 2014. For more pictures from AEA 2014, visit AEA’s Facebook page.