Data Reflections

Monday, August 28, 2006

Spring Data Conference Keynote Speaker Announced

WSWHE BOCES is pleased to announce Daniel Pink as the keynote speaker for our 2nd Annual Spring Data Conference on March 30, 2007! Dan Pink is a best-selling author and an expert on innovation, competition, and the changing world of work. His latest book, A Whole New Mind, charts the rise of right-brain thinking in modern economies and explains the six abilities individuals and organizations must master in an outsourced and automated world. Reviewers have described the book as "an audacious and powerful work," "a profound read," "right on the money," and "a miracle." Several publications - including strategy+business, The Miami Herald, and Fast Company - named it one of the best business books of 2005. A Whole New Mind is being translated into 12 languages - and will be published across Europe and Asia in 2006. For more details about the conference, please visit
www.wswheboces.org/data. We look forward to another exciting event!

Thursday, August 10, 2006

Information Rich, Knowledge Poor

We've all heard the phrase "data rich, information poor," but now we are facing a new challenge: we're becoming information rich, knowledge poor. In the past, we've needed to make sense of all kinds of data and now it seems that every day we have a new system or tool that is purporting to turn our data into information. But what does the information really tell us?

Here's an example: once upon a time we talked about percent of students at levels 1-4. Now we can see what percent of students are at levels 1-4 over time and compared to different populations. If we see we are making progress, great - but why? Our information is helping us see what is happening in our classes and in our schools, but why do we see these results? As more "what" inputs are being presented, we might find ourselves becoming led further away from our focus on "why."

So what can we do about this issue? We need to start (or continue) to approach the data from a multivariate angle - multiple inputs leading to multiple outputs. Simple descriptive statistics don't cut it when we are looking at complex data systems. And assessment data do not show the whole story so we need to include perception data and more! Thus, I challenge all of us who have the charge of turning a mountain of information into useful gems of knowledge to consider new ways of analyzing educational data. We don't need to reinvent the wheel, we just have to sharpen our tools and tackle that mountain!

Wednesday, August 09, 2006

Using A Whole New Mind to Address the Dropout Crisis

A recent ASCD InfoBrief (Summer 2006) focused on "Closing the Gap: Keeping Students in School." Looking at some of the responses from dropouts interviewed for one of the studies cited fired my synapses and I thought of Dan Pink's bestseller A Whole New Mind.

Students in the study (Bridgeland, DiJulio, & Morison, 2006) talked about not being engaged and having no incentive to go to school. Factors that students discussed included: making school more relevant and enhancing the connection between school and work. Looking at Pink's book, one will see how, in this age of abundance, "things" aren't keeping students engaged. Sure, the technology enhancements in the class are nice (and can even be helpful for instruction when used properly) - but they are expected since they are part of daily life. What students are looking for is SYMPHONY and MEANING. Students need schools to show how what they are learning fits in with their life now and their future. Students also need to feel like they derive some meaning from being in school - and if the school doesn't set up these structures, students look to their friends (in person or in cyberspace) to determine how they matter.

I don't think this comes as a surprise - these are issues that the workforce deals with every day. Plenty of research documents that intrinsic motivators win out over extrinsic motivators if you want to reduce turnover and increase productivity. So as educators, employers, and parents, let's start using A Whole New Mind to reframe the way we address the dropout rate and focus on the needs of students as they prepare for a global future.

Friday, August 04, 2006

How NOT to Complete a Survey

Based on our office's work in analyzing perception data from surveys, we notice a lot of things that people do when filling out a survey that make the analysis difficult, or worse, make responses void. Here is our short list of reminders of how to complete a survey if you want your responses to be included in the analysis.

1) When the question asks you to circle or check one response only - check or circle only one response.

Often times, an organization wants to know your top choice or the best _____, so you are asked to choose one only. When you check more than one because you think there are 2 best or 3 favorites, then the item is unusable because the researchers cannot pick which one was the "best" of those selected.

In items than have a scale (e.g., a 6-pt agreement - strongly agree to strongly disagree), we tend to see responses where folks circle more than one (a big circle around 2 and 3). Sometimes we even see an "X" in between two numbers. These are no-nos, because there is no option of 2.5 (we'll cover ordinal vs. interval scales in a later blog), and we don't know if the respondent was really more a 2 or more a 3. So please choose one, otherwise your response is unusable.

2) If the questions asks you to rank the top 3 choices, they need to be ranked - and only up to 3.

Items that ask you to rank are different from items asking you to check all that apply. If you are asked to rank the top three (1, 2, and 3), and you just check off three options, then there is no way to know which was first, second, or third choice, rendering the response void. In one recent survey, we had to exclude 25% of responses to a ranked item because folks just checked three instead of ranking them as the question prompt asked.

The same rule applies if you rank items: 1, 2, 2 (suggesting the second and third are tied). We can use the 1, but we don't know which one of the others is really second.

3) When there is an ID code asked, please fill it in completely.

To ensure confidentiality and to attempt to allow for anonymity, we often ask for ID code (a combo of name/phone number). These codes are important for pre-post studies so that responses can be matched. If the codes are not filled out completely as per the instructions in the pre- or post- setting, the surveys cannot be matched and both the pre- and post- survey may have to be excluded from the analysis.

4) If you have comments, place them in the additional comments section or at the end of the survey.

When there are tiny comments crunched into a scale survey item line or squeezed between two categories, 1) it is difficult to read and 2) there isn't a location in the database for these comments so they may not be considered. To make sure your comment is clear and counted, it is best to place them at the end of the survey.

5) Please print!

Chances are that if a respondent took the time to respond to open ended (fill in) items, the comments are important for the analyst to include and for the organization to hear. For this reason, we ask that folks type or print their comments. There have been many comments that were illegible due to the cursive or shorthand that respondents used. This is even more critical if you are scanning or faxing a document.

*****

We hope that if you see yourself in any of these comments that you are not offended and now realize why survey responses are asked for in particular ways. If you have a question about how to complete the survey, there always should be a contact person listed on the survey or on the survey cover letter (if applicable). I know our office would prefer phone calls to ensure valid responses instead of having to code responses as "missing data."

So what have you been doing all summer?!

Long time, no see. What do data analysts do all summer when there is no new state assessment data? The answer is - a lot! Most of the summer has been spent preparing for the rollout of NYSED's Grow Net (aka nySTART) reports. We will be distributing a DVD for all participating districts that includes training on how to interpret the new parent reports, we are providing "Account Provisioning Boot Camp" to assist districts in the setup of nySTART accounts, and in October/November, we'll host FALL DATA DAYS to help staff interpret the information in the online reports. Check back on the WSWHE BOCES data website for all the details. Oh - and CIO meetings are right around the corner...

We've also been busy with various administrator trainings and workshops. The WSWHE BOCES administrators worked for several days on determing how to help address 21st needs and close the achievement gap as we craft "A Whole New Mind" - see DanPink.com. We worked with Granville on moving from "Good to Great" - see JimCollins.com. And I even learned a few things at the Model Schools "Administrators' Technology Summer Camp" (although I am still looking for the s'mores)!

In addition, we've welcomed our new Director of School Support Services, Tony Muller, and we are preparing to welcome Dr. Janie Cornell to WSWHE BOCES as the new Asst. Superintendent for Instruction/SSS. Katie Jones, WSWHE BOCES Data Analyst, also welcomed a beautiful daughter, Finley Cole, to the world on July 7. We've missed Katie all summer and look forward to being back to full strength in September.

In the meantime, Rose and I are holding down the fort. We're working hard to prepare for next March's data conference - "Data Analysis in the Flat World." The call for proposals will be out in September and the planning committee is looking forward to seeing a lot of great sessions offered.

We hope you enjoy our blog and visit (and even post comments) often!