- Client feedback is important: Take time to hear your client’s thoughts on the visualizations you are creating, you want to make sure you are meeting your client’s expectations.
- Patience is key: I spent a lot of time creating and re-creating multiple charts and graphs. It takes time to make sure every visual aligns with the story you are trying to tell. If you need a break, eat a muffin.
- Don’t be afraid to ask for help: If you get stuck, ask a colleague or check out online forums to see if there is a solution to the problem you’re having. Google is a great friend.
- Sketch!: Take time to sketch out what you want your visuals to look like. Trust me, it’ll save you a lot of time in the end.
- Don’t be afraid to try: I made about 40-50 visuals for this project. Around half were rejected by the design team but I learned a lot throughout the process:
- Pie charts are not your friend
- No one at CRC likes pink or mustard yellow
- Embrace white space
- Not every visualization needs to be a bar graph
- Embrace awesomeness
From a public health standpoint, the medical benefits of breastfeeding are well established.* Breast milk provides babies with all the necessary fats, proteins, and vitamins they need for healthy growth and development. Among other benefits, antibodies in breast milk can help babies fight infections and reduce the risk developing asthma and allergies. Moms who breastfeed experience benefits too. Breastfeeding can help mom lose her pregnancy weight (through the calories it burns) and can protect her against breast and ovarian cancers. In addition to the physical benefits, time spent breastfeeding also helps nurture the bond between mom and baby.
See the full infographic here: http://www.crcdataviz.com/#!breastfeeding-in-the-usa/ccrt
* Although the above benefits of breastfeeding are established in current research, we would like to acknowledge that other bodies of research demonstrate that babies who are not breastfed also can have healthy outcomes and bond well with moms. Many moms cannot or may not choose to breastfeed for a wide variety of reasons, and we support all moms in their choices.
Some of you have been following our interest in maps at the Baltimore DataMind blog, but to make sure more of our readers get to see that content, starting with this post we’re “folding” the BDM blog in here. So now, along with the evaluation news, data tricks, and dataviz tips you’ve come to expect from CRC’s blog, expect to learn more about making and using maps. We also hope to show you a lot of just plain cool ones, starting with this one, created by our own Matthew Earls.
Baltimoreans’ love of festivals is possibly even greater than our love of maps! This map uses an interactive and chronological format to map out all the festivals in-town this summer. Use it to map out your summer plans!
Infographics are all the rage. They are beautiful, engaging, and fun to look at. This one is no exception:
At first glance, it looks like a lot of fun. The Finance Office has done something unexpected, which is trying to make understanding the budget of Baltimore City a bit easier by using data visualization. It is likely to be successful in that more people will look at this than might read a website that breaks out funding by categories, or talks about the property rate; but, it also leaves a lot to be desired.
Here are a few things we would suggest to the City of Baltimore for next time:
Be careful with them. Unlike other data visualization folks, I’m not completely opposed to the pie chart in all situations. But, if you are going to use them, you should have the slices go from the largest (property taxes at 32%) to the smallest (other at 4%) slices. This one does work okay because you don’t have a million slices.
The colors seem randomly chosen (except for the green tree and blue water) and the color palette on the top half of the infographic looks unrelated to the bottom half. Pick a few colors and stick with them throughout so the piece looks consistent. Consider gradient colors (i.e., a color fill that gradually blends from one color to another) for the Priority Outcomes.
The bottom graphic is hard to read. It might be the case that the color of the Priority Outcome is supposed to match the color of the pie slice, but it’s not clear. Are the colors communicating information or if is this just the color palette? If the former, you made a good choice as a way to unify the piece, at least the bottom of the piece. However, a bit of labelling of the bottom pie chart (like the top one) would have cleared this up.
And… What’s with the arrows? What are they pointing at?
We applaud the Finance Department for using data visualization to make the budget more accessible to citizens. And we love the clean look and feel of this piece. We’d love to see them use some of the best practices in the field of data visualization to make it even more impactful!
As a final note, thank you for NOT putting the City seal on this.
Last week I was at the Interaction Design conference. Now, you probably know that evaluation and design don’t exactly go hand-in-hand, so I understand if your next thought is a befuddled, “Huh?” You don’t usually find evaluators at a design conference. So, was I just hopping a plane to San Francisco in February because of the awesome weather and the cool town? Well, not exactly, although I have to say those were both nice perks of being there. Did I travel to San Fran for some much needed R&R and to clear my head? Not entirely, but I do always feel inspired and innovative when I’m there. (And it’s a bit surreal to go by the Uber headquarters in your Uber.) The truth is, I’ve been doing a ton of reading and thinking about design lately. The more I learn, the more I find that there is a common ground between the way we do our work here at CRC and the work of design firms. I’ll be writing a series of blogs in the future about how we use design thinking in our approach to research and evaluation, but for now I’ll just quote Phi Hong Ha, who said in her conference talk, “Design is about helping people to make sense of the world.” That’s what we evaluators do, too! Help make sense of the world. It’s our job to help people understand what is going on, what their clients are thinking and feeling, and what the reality is of the communities they work in and what the impact is of their programs. Design thinking, combined with CRC’s love of beautiful data, makes a design conference the perfect place for me to learn new things, embrace old ways of doing things with a new language, and continue to be inspired. As Tim Brown (Tim Brown!) said, “Information is the material we are most using in design today.” If that doesn’t sound relevant to what we do here at CRC, I don’t know what does. So, what did I get out of this conference? Creative innovation is risky. Smaller firms and individuals are often able to innovate, try new things, and “play” more than larger firms because we smaller firms have a higher tolerance for risk and less bureaucracy to fight.One barrier you encounter to creativity, as Dan Saffer pointed out, is that “Efficiency is the enemy of creativity.” Creativity takes time. And it’s not always linear. Researchers love linear—linear relationships, linear regression. Here at CRC we have a certain tolerance for a bit of meandering if it means a better process, a better relationship, and a better outcome. Things never turn out like you think they will. So stop expecting them to. It’s been our experience that projects never go exactly as intended. That’s because our work involves people, not widgets (or pharmaceuticals). And people change, make decisions, and adapt constantly. Jan Chipchase said, “Once you begin, assume that everything you planned is not relevant.” While this is an extreme perspective, I have found it very helpful to let go of the idea that nothing will change. Being flexible in our thinking has worked for us very well, and it’s a constant in the design world. Silicon Valley is full of people telling you failure is great. “Fail fast” they say. Foundations especially struggle with this. They often hold onto approaches or initiatives long after they should let go and move on to new and innovative ideas. Everyone struggles with change, individuals and organizations. But Saffer made a good point. “Failure sucks!” he said. Learning from failure is where it’s at. We are constantly trying out new approaches and new tools. Some of them succeed, some of them fail. But I have yet to try out something new that we didn’t learn from, whether it was software, process, or research methods. Another theme of design work is empathy. The idea of empathy often makes researchers uncomfortable. We embrace empathy here at CRC. Watching people work with their clients or fill out forms or struggle with databases gives us empathy for how we might develop solutions to make their work easier. Indi Young did a great job of defining empathy in a way that I think even evaluators and researchers can embrace. She differentiated between “emotional empathy” (feeling what someone is feeling) and “cognitive empathy”, which is about understanding how people are thinking and why and how they react. I would argue that we need some of both because we also need to understand the emotional state people are in when we are asking them to fill out forms, take attendance, complete surveys, etc. Sometimes they are upset, frustrated, or anxious (forms can bring it out in the best of us). But cognitive empathy applies too. How do people see and understand our data collection tools? How are they interpreting the questions we ask on surveys (hello, cognitive interviewing)? What makes them skip over some parts of the forms they fill out? Observing and talking with people about these things specifically can be very fruitful. Danielle Malik wins the award for best presentation title with “Go Home Data, You’re Drunk.” She talks about how future trends will be all about analytics and customization (even more so than now). Because of the visual storytelling we do here, I was excited to hear “data points are the words, and it’s up to us to construct the sentences.” Her presentation resonated with me, as she focused on the fact that data is not neutral. You have to constantly think critically about where it comes from, why you collect it, and how you intend to use it. And we’ll definitely be looking for ways to use the hashtag from her presentation … #drunkdata. I think evaluators have much to learn from the design community. Design is user centered, process oriented, and collaborative. The design process requires that you empathize with your users, understand what their problems are, come up with creative solutions, test them, and then build in a process of iteration and tweaking until you get where you need to be. I think evaluation should be more like design. I will admit that I often have doubts about the way we design and implement evaluations in this field. Evaluators often do not take the users into account. Some evaluators work in virtual isolation from their end-users. How many actually spend time in the schools, public health clinics, and program sites where their data collection takes place (besides us, of course)? How many actually talk to the people filling out the surveys to find out what they think was meant? Karl Fass, professor of User Interface design said, “We shouldn’t be using the vocabulary of natural science,” and I agree. I often question why we use the vocabulary AND the methods of the natural sciences in a contest that is very human-centered and (let’s face it) often chaotic. So I leave you with these questions…. What would happen if we used human-centered design principles and user testing to develop and evaluate social programs instead of evidence-based practices?  What if we began every program implementation with empathy, planned it collaboratively with the end user in mind, and assumed that some iteration would take place before it got to where it needed to be? What if we conducted our evaluations not like science experiments but like user analytics? What if we were careful to collect only what we (or our clients) needed, to constantly review the data and collaboratively make decisions with it, and to not assume there was a beginning or an end point? How revolutionary would that be? Well, if I’m going to be truthful, we know that good evaluation practices do these things, even though we don’t always like to talk about it. But in my upcoming posts on design thinking, I plan to do just that.  It’s a radical notion, but it’s one worth considering. Large, elaborate studies are conducted using the principals of the natural sciences. Then we plop those programs into a variety of locations as if context doesn’t matter. Designers know that context always matters. And that users can be almost infinitely segmented.
Welcome to CRC’s monthly series of articles on all things techie: Secrets from the Data Cave! (For those who don’t know, the title references our room — fondly referred to as “the bat cave”— where data staff can geek out in an isolated setting.) Here we’ll be offering you a sneak peek into the cave, with tips and the latest updates on what we’re implementing here at CRC.
(This month’s SDC is an on-the-road edition!)
Late last fall, I had the opportunity to go to phpworld 2014, a conference for PHP code developers, held in Washington DC.
Many different web applications use the programming language PHP, so coders have a myriad of options for deploying it (including, but not limited to, Drupal, WordPress, Joomla, and Magneto). This conference is designed to bring these communities together in order for developers to learn about how others are using this language, and get ideas for enhancing existing applications.
The conference sessions that fascinated me the most were those about web security. PHP is great for developers because it is a very powerful language, meaning you can do a lot with it. But, without the proper precautions, hackers can exploit that power over your applications and cause huge headaches.
One conference presenter gave a great example of how this might look in the real world. As context for the example, you need to know that there are certain combinations of coding language that, should a hacker type them into your application and you haven’t protected your database, the application will interpret them as legitimate SQL code and “drop” all the tables in your database. Therefore a hacker could use this technique (called “SQL injection”) with a “drop database table” command – feeding the application some code that masquerades as part of the original developer’s code— to irreversibly erase all of your data.
With this in mind, consider the presenter’s example: A would-be hacker in Europe rigged up a fake license plate with a bit of SQL injection code on it. He was exploiting the fact that traffic cameras use picture-parsing technology to break down an image of a license plate into individual characters (to record the plate info of speeders).
So, this was presumably an effort to trick the traffic camera into parsing the code, feeding the data in as it would any license plate. This would have ultimately caused all the stored traffic data on the back-end to be wiped clean via the SQL injection! Now, it’s unlikely that this actually worked in practice. But, it did get me thinking about the lengths hackers will go to in order to mess with your data.
All in all, it was an excellent conference, and I feel I learned a lot about this powerful coding language. I’m now looking forward to connecting with more PHP developers in future!
Technology is always changing and evolving, and data visualization technology is no exception. While the term “infographic” was once unfamiliar to most people, fun visualizations are now regularly spotted on Facebook and Twitter feeds for both individuals and businesses. But just how much can we expect to see change within a year? In the spirit of the New Year, we are posting a few predictions on where we think data viz is headed in 2015 and beyond:
Microsoft Office: it can do a lot more than you think With all the integrated apps and changes in Office 2013, you can really accomplish some amazing visualizations using nothing more than your trusty old Microsoft Office software. But the fact is that a lot of people don’t use it to its full potential, and don’t realize what it can do. We predict that in 2015, more people will figure out that Office can be used to create some really interesting visuals, like the chart of weather patterns below (an Excel template): or this dashboard we created:
No more giant reports sitting untouched in file cabinets Or, at least, fewer of them: in 2015 we predict a small but perceptible decline in the popularity of the traditional final paper report for program evaluation (you know, that one you meant to read, but never actually got around to, and then you lost it in your desk drawer?) With the rising popularity of infographics, dashboards, and even graphic memos to communicate findings, people will begin to gravitate towards these options for reporting program outcomes, as they are sometimes a better fit.
Less is more Data viz gurus like Edward Tufte have been touting this advice for a long time, but it is our hope and prediction that more people this year will come to see that less is more when it comes to visualizing data using charts and graphs. Fewer colors, borders, and dancing bears can, in fact, make your data viz better (see this concept come to life at http://i.imgur.com/WntrM6p.gif). Have a prediction you’d like to share? Leave it in the comments below!
We’ve had an extraordinarily busy year, and are so pleased to have shared much of it with you!
This month, rather than a blog with evaluation-related info and tips, we’re once again ending our work year together by taking looks back and forward.
Along with continuing to provide high quality evaluation and data services to clients — both existing and new for 2014 — Carson Research has completed its long-term project of updating our branding and communications so that they better reflect what we do and who we serve. Heading into 2015, we hope you’ll become used to hearing from us as simply “CRC”, in addition to calling us that to save breath and/or typing time.
To quickly catch you up on our 2014, our resident data viz department, held down by Ashley, beautifully summarized our year’s activities:
But, as is our custom, we also asked all the CRCers to say a little (note: our definitions of “little” vary) something about their year. Below, our staff reflects on things they’ve experienced and learned from in 2014 and things they’re looking forward to in 2015:
- Ashley, Database Analyst
This year I learned that infographics and dashboards are AWESOME! Utilizing fun icons, cool graphs, and eye-catching colors to tell data stories for our clients has been so much fun. I am looking forward to exploring more resources for creating them!
- Dana, Research Assistant
I assisted with the implementation of a community survey and, having never done something like that before, I was able to learn a lot of new skills. Planning, implementing and analyzing the results of the surveys within a 2-3 month span was hectic to say the least, but the experience helped me gain a lot of valuable tools and resources. I also took a qualitative analysis course at The Evaluator’s institute in DC and an AEA online webinar on survey methods and sampling, which helped me learn new tips and tricks that I can incorporate into my current and future work. I also had the opportunity to learn some Arc GIS mapping skills and self-taught myself some rudimentary VBA coding skills on excel as I worked on a proposal database for CRC. I’m looking forward to working on new projects and learning new things from them, as well as utilizing some of the skills I have learned over the course of this year, such as mapping.
- Jill, Research Analyst
In 2014 I was greatly relieved and proud to officially complete my PhD program at UMBC! My favorite learning opportunity of the year was attending Museum Camp at Santa Cruz Museum of Art and History. It provided me with inspiring and invigorating ideas about how to make evaluation more user-friendly, engaging, and even fun for everyone involved! Next year I hope to apply more of what I learned at camp, attend other, similarly enlightening conferences, as well as to continue honing my visualization skills.
- Kevin, Controller
I tore my ACL in July throwing discus and had to drop out of track & field at the 2014 Gay Games. I postponed surgery and went on to win 3 gold and a bronze in my main sport of swimming at the Games. I am now rehabbing after cadaveric ACL replacement & meniscus surgery. I still have many months of PT ahead of me but have registered for the 2015 EuroGames in Stockholm, Sweden to be held in August. It’s on!
- Leslie, Research Analyst
This fall, I began a second master’s degree in Educational Psychology. It’s been challenging fitting studying into my schedule and the online format of the program is a new experience for me, but I enjoy being back in school. I’m looking forward to the classes I’ll take next semester in educational measurement and assessment development.
- Mandi, Research Assistant
What I learned this year: (1) Giving presentations isn’t so scary after all – even at national conferences. (This year I attended AEA 2014 as a first time presenter); (2) Focus groups with can be both fun and informative. (I performed a number of child focus groups this past year and found there were certain activities that kept kids more engaged during the session than others. Stay tuned for an upcoming CRC blog for details!). What I’m looking forward to next year: (1) More focus groups (I’m looking forward to conducting more focus groups next year, refining my interviewing skills, and finding different focus group activities that are both fun and engaging.); (2) Data visualization (I plan to incorporate more data visualization into the work I do. Including qualitative data visualization tools and techniques; (3) Attending conferences (I enjoy the workshops, networking with other evaluators, and traveling, especially to places I’ve never been to before).
- Matthew, GIS Analyst
This year I began learning how to code and to use that code to make more dynamic and customized maps. Next year I’m looking forward to expanding GIS skills and coding to grow the mapping services division here at CRC.
- Sarah, Technology & Information Specialist
The coolest thing that I learned this year was how to program in PHP. It’s an incredibly powerful programming language! And next year, I am looking forward to launching our own online database application, using PHP and some other fun things!
- Sheila, Research Analyst
The coolest thing I did this year was travel. At EERS, it was great to see old friends and hear Dr. Rog’s talk on infusing evaluation theory with practice. In New Orleans, hearing six surgeons general speak at APHA about their challenges, accomplishments and vision for the future was inspirational. In Detroit, I got to learn how Shinola makes their watches; it was also great to visit D3 and see how the organization works to provide communities with to high-quality data. In Minneapolis, I learned about different data viz techniques at EYEO and visited art installations at Northern Spark. In New York, I attended the launch of Data & Society and met great folks who are researching the social, cultural, and ethical issues arising from data-centric tech development. At the CIC impact summit in DC, I heard how various tools can be used to tell more meaningful, accurate, and connected stories that can improve community outcomes. All were great experiences, but watching world renowned rugby players compete for the World Series Title at Sam Boyd Stadium was, by far, the most awesome thing I did this year. What’s next? In 2015 I’m looking forward to perfecting my chocolate mug cake recipe, learning more about data viz and improving my mapping skills. I’d also like to tackle learning a new language on duolingo.
- Taj, CEO
The big game changer for me was attending the EYEO Festival in Minneapolis in June. EYEO brings together data geeks, coders, designers and artists for some of the most fascinating presentations and conversations I have ever experienced at a conference. It is THE data visualization gathering, in my opinion. I was so inspired by my experiences there, that I chose to take the big leap to go back to school. I’m now back in grad school, earning (another) Master’s Degree, this time in Information Visualization at MICA. Becoming a beginner again is very humbling, and I’m excited to see how what I learn in this program can help CRC go so much farther in our efforts to help our clients to visualize and understand their data.
- Tracy, Research Associate
My biggest learning experience in 2014 was designing and implementing a community survey in two Baltimore City neighborhoods. In the coming year, I anticipate opportunities to work with new clients and expand my evaluation skill set.
Happy New Year, and here’s to new learning, growth, and gainful opportunities… as well as more beautiful data… for you and yours in 2015!
Last month I attended the 2014 American Evaluation Association conference in Denver, CO. 2014’s conference theme was “Visionary Evaluation for a Sustainable, Equitable Future.” The event brought together research and evaluation professionals from all over the globe and from a variety of disciplines (e.g., community psychology, health and human services, PreK-12 educational evaluation). Attendees were encouraged to explore ways in which evaluation could be used to support sustainability and equality across disciplines and sectors.
This year’s conference was especially exciting (as well as nerve-wrecking) for me because I was attending as a first time conference presenter. I went to numerous sessions, learned a lot, and had a great time connecting with other evaluators. (I even found a little bit of time to explore Denver’s spectacular shopping scene). Below are some of my highlights from the conference.
- Robert Kahle: Dominators, Cynics, and Wallflowers: Practical Strategies for Moderating Meaningful Focus Groups
- Veena Pankaj: Data Placemats: A DataViz Technique to Improve Stakeholder Understanding of Evaluation Results
Deciding to one-up myself by giving not only one, but two presentations my first go around didn’t really help my nerves, but what can I say, I was enthralled by this year’s conference theme (you could probably also say there was a little part of me trying to impress the boss as well). I gave a poster presentation on an infographic that we (CRC) created for Elev8 Baltimore in effort to visually display evaluation findings. The poster reviewed the process, results, and implications of translating data findings into a reader-friendly infographic.
My poster implied that infographics can be successfully translated into attractive, functional, and informative infographics. It suggested the use of infographics to report evaluation findings, as effective data visualization can attract readers, aid in interpretation of data, and support comprehension. Perhaps most importantly, it implied infographics can be used to promote the use of evaluation findings to inform decision making!
I also gave a paper presentation on our (CRC’s) establishment of an early warning system at the Elev8 Baltimore sites, and reviewed the application of early warning indicators to the middle grades. Within the presentation I gave a brief description of the system and reviewed the steps we took to create it – including everything from gaining access to the data to producing the final reports. In this presentation I described how early warning indicators can enhance the accessibility and use of evaluation data; through providing reports to sites on a quarterly basis, the early warning indicators can be used on a real-time basis to inform programmatic decisions. Implications were also made for expanding the use of early warning indicators from high school to middle school populations.
Between the multitude of great sessions I attended, learning loads of information, giving two presentations, and still finding time to scope out the shopping scene in Denver, you could say I had a whirlwind of a time at AEA 2014. For more pictures from AEA 2014, visit AEA’s Facebook page.
Welcome to CRC’s monthly series of articles on all things techie: Secrets from the Data Cave! (For those who don’t know, the title references our room — fondly referred to as “the bat cave”— where data staff can geek out in an isolated setting.) Here we’ll be offering you a fascinating sneak peek into the cave, with the latest updates & tips on what we’re implementing here at CRC!
Visualizing Nonprofit Data: Tell the Real Story by Using Your Program Knowledge
(This blog post originally appeared last month as a guest post for the Maryland Association of Nonprofit Organizations.)Many nonprofit organizations rely on in-house staff members to crunch numbers and create reports for their program data. This means that, in some cases, those who are inexperienced at turning heaps of data into charts and graphs will find themselves stuck with a daunting task: creating meaningful data visualizations for their program. Now, there are a few basic tenets of data visualization to which everyone should adhere. For example, all pieces of a pie chart need to add up to 100%. But let’s say you know the basics already. What else might you want to keep in mind when choosing a visualization for your data? My top suggestion: never forget that YOUR knowledge of your program and the populations you serve can make all the difference when it comes to presenting your results. So, if you think your data visualizations are not showcasing your program results in the way you expected, try to determine why they might look this way. There’s more than one way to visualize data. Let’s look at a fictional program as an example: say I am reporting on outcomes a program serving adults struggling with addiction. I want to show that clients who participate this program are not only more likely to start drug & alcohol treatment, but also to successfully complete the treatment program as prescribed, than those in a control group of equal size (who did not participate in the program).When I look at the results, however, I am disheartened. It appears as though there is very little difference between the two groups in terms of who is more likely to complete treatment: Yet anecdotal evidence from my fictional program (talking to program participants and non-participants, etc.), tells me that those in the program do indeed seem more likely to overcome their addictions, stay in treatment, and stay clean. So what went wrong?Well here’s something I had not considered: what if most of the addicts concerned in both groups cited heroin as their drug of choice. In this case, their prescribed drug treatment would likely be an opioid maintenance program, i.e. methadone maintenance therapy (MMT), as this is commonly used in the treatment of opioid dependence. MMT is a treatment that can go on for years, in some cases. So the individuals on MMT would not technically be considered to have “Successfully Completed Treatment,” since they are not yet finished with treatment, but many have abstained from drug use for the entire time and have reformed their lives such that had they not been on MMT, they would be considered “Successfully Completed.” To remedy this, I use the same pie charts—but this time, I give a breakdown of the portion of clients that did not successfully complete treatment to show those who are still in MMT: This visualization shows that while the percentage that technically completed treatment is almost the same for the two groups, a large portion of program participants who did not complete treatment are still on Opioid Maintenance. In the non-participant group, the vast majority of those who did not complete treatment had “other” reasons for this (like dropping out of treatment early). This gives a completely different perspective on the same data, and shows just how powerful your choice of visualization can be. While both examples are accurate representations of the data, one is simply more effective at showing the program results.
So remember, YOU know YOUR program better than anyone else does. Use that knowledge to choose a visual representation that shows the world what your program has achieved!