Noisy Measurements Are Important, the Design of Census Products Is Much More Important

Key Question Being Explored in This Insight:

John M. Abowd discusses that the more important component is the query workload output--the statistics released to the public.

View the page of: John Abowd

Institution: Cornell University

Explore more insights in:

Econometrics Economics Social Sciences

Media Literacy Toolkit

Faculti's Media Literacy Toolkit helps viewers critically engage with academic insights by analyzing the research context, identifying perspectives, and encouraging thoughtful evaluation.

Critical Questions to Consider ▼

What assumptions does the research make?
Are there alternative perspectives not explored?
What are the limitations of the research method?

Bias and Perspective Awareness ▼

This research comes from Cornell University . Reflect on how the institution's academic focus and research partnerships may shape the questions being explored.

Further Reading ▼

Read the Study

John M. Abowd discusses that the more important component is the query workload output–the statistics released to the public.

Image courtesy of interviewee. March 15, 2024

Log-in or Sign-up to Faculti
Currently viewing this subject insight as a guest. You have insight(s) remaining for this month. Login to view 8000+ figures on the platform.

Generating Workspace...

Export as PDF

This video video discusses the complexities and methodologies involved in the US Decennial Census, specifically the 2020 Census, which had a significant budget of approximately $15 billion. The Constitution mandates that the census produces data for various purposes such as the apportionment of seats in the House of Representatives, which is based on population counts derived from census data.

This video video data is also used for redistricting voting districts at both national and state levels and must comply with the Voting Rights Act, ensuring fair representation and adherence to the principle of "one person, one vote." There are statutory obligations for producing demographic estimates annually, which requires detailed data analysis for resource allocation.

This video video explains the disclosure avoidance system implemented for the 2020 Census to protect the confidentiality of respondents while allowing for the publication of various statistics, including race, ethnicity, housing, and population characteristics. This video video system employs a differentially private framework, applying noise to make the data less identifiable while preserving essential aggregate information.

Five key constraints were established for the system: ensuring data for apportionment wasn't affected, specifying data publication formats preemptively, limiting confidentiality protections to the final census version, integrating into the previously designed tabulation system, and ensuring timely publication.

The process generates a vast number of "noisy measurements," with significantly more measurements than the statistics published, allowing for detailed querying capabilities. Researchers can access these noisy measurements, which can provide greater detail than published tables but may introduce some biases.

Finally, the text emphasizes the ongoing challenge of balancing data usability for the public while maintaining the confidentiality required by law, a tension also faced by tech companies working with sensitive data. As the landscape of data privacy evolves, statistical agencies will need to refine their approaches to managing this delicate balance.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:05.065 --> 00:09.875
The US Decennial Census is a,

00:11.105 --> 00:12.915
very large data collection effort.

00:13.455 --> 00:16.835
The budget for the 2020 census was about $15 billion.

00:17.775 --> 00:19.395
The Constitution

00:19.395 --> 00:23.955
and statutory basis for, for that census requires it

00:23.955 --> 00:26.355
to produce data for some purposes.

00:26.975 --> 00:30.075
So it requires it to produce data for apportioning,

00:30.575 --> 00:33.195
the House of Representatives, which in the United States

00:34.095 --> 00:38.515
is the part of our, two Body Congress that is,

00:39.215 --> 00:41.715
representation based on population.

00:41.735 --> 00:44.595
So in principle, every member of the House

00:44.595 --> 00:47.355
of Representatives represents the same number of people.

00:48.095 --> 00:52.035
And, and so the apportionment of seats in the House

00:52.035 --> 00:55.875
of Representatives is, based on a formula.

00:56.015 --> 00:58.235
And that formula uses population counts,

00:58.255 --> 01:00.835
and they, by constitution have

01:00.835 --> 01:02.435
to come from the decennial census.

01:02.975 --> 01:04.555
So that's the constitutional basis.

01:05.295 --> 01:06.795
The next, the next activity

01:06.795 --> 01:09.195
that happens is those same data are used

01:09.855 --> 01:13.915
to redraw all the voting districts in the United States,

01:14.295 --> 01:16.835
voting districts for the Congress, which is a national body,

01:17.015 --> 01:19.155
the voting districts for state legislatures,

01:19.445 --> 01:20.875
which are usually,

01:20.935 --> 01:24.075
two body bicameral like the federal government, except

01:24.075 --> 01:25.555
that both of those bodies are required

01:25.555 --> 01:27.235
to be one person, one vote.

01:27.455 --> 01:30.355
Um, in addition to that,

01:31.555 --> 01:35.035
redistricting requires that the data, that the districts

01:35.695 --> 01:40.155
comply with the 1965 Voting Rights Act,

01:40.405 --> 01:44.235
which has some very specific requirements in section two

01:44.805 --> 01:48.395
about what constitutes a legal redistricting map.

01:48.535 --> 01:51.595
And so the data are used to support that,

01:51.595 --> 01:54.195
and that's where the reason of ethnicity, data come in.

01:54.195 --> 01:55.875
That's what they're used for. Now,

01:55.875 --> 01:57.955
that's also a statutory obligation.

01:57.985 --> 02:01.795
Finally, the, the statute says the data should be used

02:02.015 --> 02:06.235
to support the, estimation of population

02:06.235 --> 02:09.635
and population characteristics every year in what's known

02:09.635 --> 02:12.195
as the Population Estimates program at the Census Bureau.

02:12.375 --> 02:15.235
So that's a, a, a requirement

02:15.235 --> 02:18.755
to produce more detailed demographic data on,

02:18.935 --> 02:20.155
on the population.

02:20.155 --> 02:21.915
And it's updated every year on the,

02:22.015 --> 02:25.075
the census data are supposed to be published in a way

02:25.075 --> 02:27.635
that supports their use in, in

02:27.635 --> 02:29.195
that those population estimates.

02:29.195 --> 02:32.195
Those population estimates are then used in many different

02:32.355 --> 02:34.355
formulas for apportioning resources.

02:34.355 --> 02:39.025
That's why they're, so critical On top of these,

02:40.025 --> 02:42.185
constitutional and statutory products,

02:42.235 --> 02:45.625
there are very detailed race and ethnicity data published.

02:45.625 --> 02:48.305
There are very detailed housing characteristics

02:48.305 --> 02:49.785
and household characteristics published.

02:50.485 --> 02:53.625
Uh, there have historically been, public use micro data

02:53.695 --> 02:55.225
that will also be true in 2020,

02:55.245 --> 02:56.665
but they're in a slightly different form.

02:57.045 --> 03:00.965
And then what's new for, 2020 is the,

03:00.985 --> 03:04.565
the Census Bureau has published the raw output from the

03:04.565 --> 03:08.245
disclosure avoidance system, which, no agency in the US

03:08.585 --> 03:10.805
or or in the world has ever done before.

03:11.105 --> 03:12.845
And I'll get into the details of all that work.

03:13.225 --> 03:14.885
But that disclosure avoidance system

03:14.885 --> 03:16.845
that you referenced in the start of the question,

03:17.595 --> 03:20.045
it's designed to protect the confidentiality

03:20.505 --> 03:23.965
of the underlying census responses in a manner

03:23.965 --> 03:27.405
that allows all of these uses to be published.

03:27.785 --> 03:29.365
Uh, so it's a comprehensive

03:29.825 --> 03:31.645
confidentiality protection system.

03:32.345 --> 03:36.765
I'm going to do a little later how some of the constraints

03:37.455 --> 03:39.165
cause the way that system works

03:39.625 --> 03:42.445
to introduce known biases in certain statistics.

03:42.445 --> 03:45.525
But, basically it doesn't have anything to do

03:45.525 --> 03:47.325
with the differential privacy framework.

03:47.745 --> 03:49.445
It has to do with the constraints

03:49.445 --> 03:51.725
that were put on the design of the system in order

03:51.725 --> 03:54.365
to make it meet these, constitutional

03:54.485 --> 03:58.565
and statutory requirements in order to protect

03:59.145 --> 04:02.285
the 2020 census confidentiality,

04:02.285 --> 04:04.045
which is a statutory requirement.

04:04.045 --> 04:05.885
It's in the Census Act for the United States.

04:06.545 --> 04:09.605
And to meet the use cases that I just described

04:09.605 --> 04:14.005
for publication, the senior executive staff at the Census

04:14.005 --> 04:18.445
Bureau, I was on that board, put five constraints

04:19.225 --> 04:20.925
on what the system had to do.

04:21.905 --> 04:23.645
So the first thing the system had to do

04:24.225 --> 04:27.285
is it could not affect the apportionment use case.

04:27.425 --> 04:31.125
So nothing about the confidentiality protection could affect

04:31.425 --> 04:34.365
the populations that are published for the 50 states

04:34.385 --> 04:36.325
of the District of Columbia, as they're used

04:36.325 --> 04:37.725
to apportion representatives

04:37.725 --> 04:38.805
in the House of Representatives.

04:39.465 --> 04:40.645
DC doesn't actually get any,

04:40.665 --> 04:42.445
but its number does get published.

04:43.145 --> 04:44.885
Uh, the District of Columbia doesn't get any.

04:45.305 --> 04:48.325
So, that's constraint one.

04:49.615 --> 04:53.605
Constraint two is the way

04:54.385 --> 04:58.885
the publication is set up for the purposes of redistricting

04:59.585 --> 05:01.085
has been pre-specified.

05:01.635 --> 05:04.125
It's pre-specified in a negotiation

05:04.125 --> 05:05.405
between the Census Bureau

05:05.785 --> 05:08.845
and a non-partisan group known as the National Conference

05:08.845 --> 05:12.205
of State Legislatures pre-specified in a way as to

05:12.635 --> 05:17.125
what this geographic area is that the data have to apply to.

05:17.425 --> 05:18.605
So that's called an atom.

05:18.745 --> 05:21.005
Um, that geographic atom is something known

05:21.005 --> 05:22.045
as a census block.

05:22.425 --> 05:24.005
And those census blocks are used

05:24.185 --> 05:26.685
to form all the voting districts of all

05:26.785 --> 05:29.805
of the legislative bodies, state, county,

05:30.405 --> 05:31.845
national, in the entire country.

05:32.995 --> 05:35.405
They do not have a population limit.

05:35.675 --> 05:38.845
They can be as small as zero, some are as big

05:38.845 --> 05:42.245
as a thousand people, but they're pre-specified.

05:42.245 --> 05:45.045
They've already been drawn, they were drawn ahead

05:45.045 --> 05:47.925
of taking the census, and so they can't be changed.

05:47.925 --> 05:50.165
When you find that one doesn't have any population,

05:50.185 --> 05:52.365
and it just is published with zero population.

05:53.185 --> 05:57.085
In addition, the way that those data tabulate the race

05:57.225 --> 06:00.245
and ethnicity of the respondents to the census

06:00.905 --> 06:02.045
is pre-specified,

06:02.315 --> 06:06.615
because the minute those data are published,

06:07.155 --> 06:09.935
the redistricting offices in the States

06:09.955 --> 06:11.975
and the United States ingest them

06:12.075 --> 06:14.095
and start drawing redistricting plans.

06:14.245 --> 06:16.575
They have redistricting plans available the next day.

06:17.035 --> 06:20.295
So everything about those data has been pre-specified,

06:20.675 --> 06:22.735
and it was pre-specified in 2018.

06:22.955 --> 06:26.015
So those were constraints, they couldn't be changed.

06:26.795 --> 06:30.535
Um, the third constraint was that the only thing

06:30.535 --> 06:32.695
that the confidentiality protections could apply

06:32.695 --> 06:35.415
to was the final version of the 2020 census.

06:35.605 --> 06:37.615
It's known as the census edited file.

06:38.115 --> 06:40.655
So that's not just the record of what people said.

06:41.125 --> 06:44.615
It's also all of the edits that were used

06:44.615 --> 06:47.975
to make those responses usable data, filling in,

06:47.975 --> 06:52.655
missing data and other corrections, consistency corrections.

06:54.355 --> 06:57.855
The fourth constraint was that it had to deliver its output

06:58.675 --> 07:01.535
to the system already designed

07:01.675 --> 07:04.735
and implemented to publish the 2020 census known

07:04.735 --> 07:05.815
as the tabulation system.

07:06.315 --> 07:09.975
So the tabulation system had to ingest what are known

07:09.975 --> 07:14.215
as micro data, a record level image of each of the responses

07:14.235 --> 07:18.055
to the 2026, each household, and each person has a record,

07:18.435 --> 07:21.415
and that's what the tabulation system expected as input.

07:21.685 --> 07:25.375
Finally, the fifth constraint is the system couldn't delay

07:25.375 --> 07:27.175
the publication of the redistricting data.

07:27.175 --> 07:28.415
They had to be published on time.

07:28.715 --> 07:29.935
Now, as it turns out,

07:29.935 --> 07:31.695
because of the pandemic, they were

07:31.695 --> 07:32.895
published four months late.

07:33.515 --> 07:35.615
But, that was because of the pandemic.

07:35.715 --> 07:38.215
The, the disclosure avoidance system,

07:38.615 --> 07:41.535
operated within its time budget, to allow that.

07:42.075 --> 07:46.215
So the way it works is it imposes a consistent

07:46.375 --> 07:48.255
differential privacy framework.

07:48.725 --> 07:49.935
That framework is known

07:50.035 --> 07:52.495
as zero concentrated differential privacy.

07:52.965 --> 07:56.055
It's an accounting framework for every statistic.

07:56.365 --> 07:59.575
There's precise measure of how much

08:00.515 --> 08:04.895
it increases the disclosure risk of the data publications,

08:04.955 --> 08:09.255
and you can add those, disclosure assessments up.

08:09.715 --> 08:12.175
That's the single biggest advantage

08:12.315 --> 08:13.855
of differentially private systems.

08:13.925 --> 08:15.735
They provide this accounting mechanism

08:16.085 --> 08:19.735
that lets you not guess at how disclosive the statistic is

08:19.875 --> 08:23.015
and not guess at how disclosive the whole package is.

08:23.355 --> 08:25.015
You have measures for both of those.

08:25.795 --> 08:28.935
And then it uses a mechanism, which is a specific way

08:29.275 --> 08:31.175
of adding random noise

08:31.875 --> 08:34.655
to the confidential statistics before they're published.

08:34.885 --> 08:37.855
That mechanism is known as the discrete Gaussian mechanism,

08:38.085 --> 08:41.255
meaning it draws something that looks like in statistics

08:41.255 --> 08:42.775
of normal random variable,

08:42.775 --> 08:45.055
but normal random variables are continuous.

08:45.555 --> 08:47.855
You can't use continuous data in these systems

08:47.965 --> 08:50.495
that get compromises their implementation.

08:50.755 --> 08:52.295
You have to use integers.

08:52.635 --> 08:54.735
And so if you wanna use integer, then you

08:54.735 --> 08:57.285
Use a known as the discrete Gaussian mechanism.

08:57.755 --> 09:00.965
That is how the 2020 census disclosure

09:00.965 --> 09:02.325
avoidance system works.

09:02.705 --> 09:05.805
It takes those noisy measurements, the output

09:05.825 --> 09:07.925
of the discrete Gaussian mechanism,

09:08.545 --> 09:12.765
and then it converts them to micro data by post-processing

09:13.385 --> 09:18.125
in an algorithm that has been specifically designed to make

09:18.825 --> 09:23.365
the micro data that gets tabulated as accurate as possible

09:23.985 --> 09:27.245
for the tabulation tables that eventually get published.

09:27.665 --> 09:30.725
So those things are all numerical calculations,

09:30.725 --> 09:31.765
they're estimated.

09:31.825 --> 09:34.205
And then, the trade-offs are assessed

09:34.225 --> 09:36.605
and you adjust the, the parameters

09:36.605 --> 09:39.605
that are under your control until you have tuned the system.

09:39.705 --> 09:42.925
And multiple demonstration products will produce to show

09:42.945 --> 09:44.085
how the system was tuned.

09:44.625 --> 09:49.205
So the top-down algorithm, the way it works is it starts

09:49.345 --> 09:51.965
by doing the best job it can with the whole country,

09:53.355 --> 09:56.855
and then it takes micro data for the whole country,

09:57.145 --> 10:00.295
which necessarily doesn't have any geographic identifiers on

10:00.295 --> 10:01.415
it because it's for the whole country.

10:01.795 --> 10:04.455
And it goes down to the next level in what's known

10:04.455 --> 10:05.735
as the geographic hierarchy.

10:05.735 --> 10:07.055
And that next level is the state.

10:07.715 --> 10:10.295
So taking the best you can do

10:10.315 --> 10:13.295
to produce micro data at the national level, as given

10:14.195 --> 10:17.095
you then produce all the state level tables.

10:17.095 --> 10:19.735
So that's, 50 states, the District of Columbia

10:19.795 --> 10:22.535
and the Commonwealth of Puerto Rico, all those states,

10:23.075 --> 10:26.215
all those tables are constrained to aggregate

10:26.215 --> 10:28.015
to the national table, but they are

10:28.015 --> 10:30.335
otherwise the best you can do state by state.

10:31.245 --> 10:33.575
Once you have that done, you pass to the next level

10:33.575 --> 10:36.495
of the geographic hierarchy, which is known as counties

10:36.595 --> 10:37.735
or county equivalents.

10:38.115 --> 10:40.655
I'm abstracting from some details here.

10:40.995 --> 10:43.335
So people familiar with the system will recognize that,

10:43.335 --> 10:45.135
but it's the best way to describe it.

10:45.475 --> 10:48.135
At the county level, you take the state level as given,

10:48.555 --> 10:50.735
and then you produce micro data

10:50.765 --> 10:53.735
that gives the best county tables that are consistent

10:53.735 --> 10:55.375
with the state, and those are also

10:55.375 --> 10:56.735
consistent with the nation.

10:57.365 --> 10:59.655
Then you pass to the next level of geography.

11:00.075 --> 11:02.295
Uh, there are, depending on which system we're talking

11:02.295 --> 11:04.215
about, the levels now get more complicated.

11:04.315 --> 11:07.095
But basically there's a tabulation area known

11:07.095 --> 11:09.935
as a census tract, which is basically the next area,

11:10.695 --> 11:13.455
a tabulation area known as a census block group,

11:13.465 --> 11:14.495
which is the next area.

11:14.755 --> 11:17.495
And then finally you get to those atoms that I described

11:17.495 --> 11:18.895
before, which are the census blocks.

11:19.395 --> 11:21.735
And in the 2020 census, there are 8 million

11:21.755 --> 11:23.215
of those census blocks.

11:23.475 --> 11:28.295
So at that point, you're optimizing 8 million table sets so

11:28.295 --> 11:31.815
that they aggregate to the about 250,000,

11:32.465 --> 11:34.975
block group table sets, which then aggregate

11:34.975 --> 11:38.295
to the about 75,000 tracked aggregates.

11:38.555 --> 11:41.335
And all of those micro data are consistent,

11:41.915 --> 11:46.215
so they can then be sent as a single record that has all

11:46.215 --> 11:48.015
of those geographic identifiers in it

11:48.015 --> 11:49.095
to the tabulation system,

11:49.225 --> 11:51.895
which then produce the tables according to the same way

11:51.895 --> 11:55.445
that users are used to using them from previous censuses.

11:55.835 --> 11:59.005
What the Census Bureau has labeled noisy measurement files

11:59.985 --> 12:04.445
are the raw output from the disclosure avoidance system from

12:04.445 --> 12:05.525
the top-down algorithm.

12:06.265 --> 12:07.565
And what that means is

12:08.035 --> 12:11.365
that the top-down algorithm operates in two steps.

12:12.935 --> 12:16.395
The first step takes these noisy measurements,

12:16.735 --> 12:18.995
and then the second step does the optimization

12:19.025 --> 12:21.355
that I just described to produce the micro data

12:21.355 --> 12:22.955
that are subsequently tabulated.

12:23.735 --> 12:27.395
So the noisy measurements themselves constitute

12:28.645 --> 12:33.135
every question that the designers

12:33.135 --> 12:37.165
of the system determined needed to be answered.

12:37.175 --> 12:39.685
Those are called the query strategy.

12:40.675 --> 12:43.005
They are put directly to the confidential data,

12:43.465 --> 12:47.245
and back comes a tabulation, a number to that number.

12:47.545 --> 12:52.405
You add the noise from the, discreet gian mechanism.

12:54.245 --> 12:56.625
The confidential number plus the noise

12:57.045 --> 12:58.225
is a noisy measurement.

12:58.965 --> 13:01.825
And there are two sets of noisy measurement files.

13:02.255 --> 13:04.065
They're the noisy measurements that were taken

13:04.165 --> 13:05.745
to produce the redistricting data,

13:06.485 --> 13:08.665
and then the noisy measurements that were taken

13:09.165 --> 13:11.025
to produce what's called the demographic

13:11.045 --> 13:13.465
and housing characteristics, DHC data,

13:13.595 --> 13:15.705
those are those general demographic data

13:15.895 --> 13:17.625
that I described, earlier.

13:18.685 --> 13:23.145
So to produce the redistricting data, 16 billion,

13:23.725 --> 13:25.025
that's a billion with a B.

13:25.835 --> 13:27.345
Noisy measurements are taken.

13:28.525 --> 13:31.505
The query strategy in order to tabulate

13:32.285 --> 13:35.025
1.5 billion statistics.

13:35.045 --> 13:37.105
The actual redistricting data are about

13:37.125 --> 13:38.705
1.5 billion statistics.

13:39.925 --> 13:44.065
The reason there are so many more noisy measurements taken,

13:44.735 --> 13:49.365
then there are, publication tables is

13:49.365 --> 13:51.245
that requirement to convert the noisy

13:51.245 --> 13:52.565
measurements to micro data.

13:53.195 --> 13:56.565
That requirement is what, means.

13:56.565 --> 14:00.325
You're going to ask many more queries than you're going

14:00.325 --> 14:01.685
to publish the answers to.

14:02.775 --> 14:05.005
Right? For the demographic

14:05.005 --> 14:09.865
and housing characteristics, there are 25 trillion

14:10.055 --> 14:12.345
with a t noisy measurements,

14:12.495 --> 14:15.465
even though there are only about 8 billion or eight

14:15.465 --> 14:18.425
and a half billion statistics, the reason

14:18.425 --> 14:20.985
that those answers are approximate is it's very difficult

14:20.985 --> 14:24.505
to count the number of linearly independent statistics the

14:24.505 --> 14:26.145
way the Census Bureau published tables.

14:26.365 --> 14:29.945
So those are, approximations that are, that are,

14:30.245 --> 14:31.905
certainly the right range.

14:32.725 --> 14:35.265
The reason that there's so many more noisy measurements

14:35.265 --> 14:38.185
for DHC than there are for the redistricting is

14:38.185 --> 14:42.785
that there's a lot of very detailed queries that have

14:42.785 --> 14:46.545
to be answered in order to publish the kinds of tables

14:46.545 --> 14:47.825
that are in the demographic

14:47.825 --> 14:48.905
and housing characteristics,

14:49.075 --> 14:52.445
which not only tell you a person's sex

14:52.545 --> 14:56.605
and, that person's, race,

14:57.235 --> 14:59.645
they tell you, their age

14:59.665 --> 15:01.885
and a variety of different, categories.

15:02.115 --> 15:04.685
They tell you things about the composition of the household.

15:04.755 --> 15:06.645
They tell you some things about the structure

15:06.665 --> 15:11.005
of the housing unit, that requires taking

15:11.195 --> 15:14.165
what we call a detailed query that has very,

15:14.195 --> 15:15.285
very high dimension.

15:15.425 --> 15:17.285
And that's why there are 25 trillion

15:17.585 --> 15:19.125
of those, noisy measurements.

15:19.495 --> 15:21.245
There are so many of those noisy measurements.

15:21.245 --> 15:22.245
They couldn't be handed

15:22.585 --> 15:25.085
to the census bureau's dissemination system to publish

15:25.355 --> 15:28.445
that didn't fit on the entire server farm.

15:29.065 --> 15:33.045
So separate arrangements were made with, with platforms

15:33.045 --> 15:36.285
that had access to, cloud computing environments,

15:36.305 --> 15:38.645
and they were to, they were deployed

15:38.665 --> 15:40.885
to cloud computing environments in order

15:40.885 --> 15:42.165
to allow them to be analyzed.

15:42.745 --> 15:45.405
Uh, so those noisy measurements have no post

15:45.405 --> 15:46.445
processing done to them.

15:46.715 --> 15:50.805
They're exactly the confidential number, plus the noise.

15:51.585 --> 15:53.685
And the reason that researchers

15:54.225 --> 15:58.165
are requesting the noisy measurements is that,

15:58.195 --> 16:00.645
there's a lot more information in the noisy measurements

16:00.755 --> 16:02.565
than there are in the published tables,

16:02.955 --> 16:06.685
because the noisy measurements had

16:06.685 --> 16:08.605
to take many more measurements

16:09.275 --> 16:11.405
than eventually got tabulated.

16:12.105 --> 16:16.045
So that's really the same thing as asking for micro data.

16:16.075 --> 16:17.565
It's just in a very different format.

16:17.985 --> 16:21.365
The 25 trillion noisy measurements from the DHC,

16:21.865 --> 16:24.885
our 21st century version of micro data.

16:25.305 --> 16:27.965
You can ask questions of those data at any level

16:27.965 --> 16:30.485
of geography that is in the,

16:30.865 --> 16:33.805
the top down algorithm for any detail.

16:33.825 --> 16:36.085
That's in what's called the detailed query.

16:36.465 --> 16:38.925
So lots of things that weren't in published tables have an

16:38.925 --> 16:40.165
answer in the noisy measurements.

16:40.195 --> 16:41.205
It's a noisy answer,

16:41.705 --> 16:43.285
but it's already been passed

16:43.285 --> 16:45.125
through the confidentiality protection system.

16:45.185 --> 16:47.485
The privacy loss has already been expended.

16:47.745 --> 16:51.285
And so the researchers can use that, to, to,

16:51.505 --> 16:53.525
to ask questions that they're not able to ask

16:53.865 --> 16:55.085
of the published tables.

16:56.515 --> 16:58.455
So that's what the noisy measurements are.

16:58.875 --> 17:02.735
It was a major innovation to publish them.

17:03.105 --> 17:06.885
There will also be a public use, file

17:06.915 --> 17:09.045
that looks like a public use micro data file,

17:09.265 --> 17:13.085
but it is actually the, the post-process micro data.

17:13.155 --> 17:15.045
It's the, the, the record that comes out

17:15.045 --> 17:16.765
of the top down algorithm, not the record

17:16.765 --> 17:18.325
that went into the top down algorithm.

17:18.665 --> 17:21.125
So that's what the noisy measurements are, the difference

17:21.125 --> 17:22.685
between the noisy measurement files

17:22.945 --> 17:25.765
and the query workload output.

17:26.025 --> 17:27.405
Let me simplify it a little bit.

17:27.665 --> 17:31.005
The query workload output is the tables

17:31.005 --> 17:33.805
that actually get published in principle,

17:34.415 --> 17:35.645
those are the tables

17:35.645 --> 17:37.445
that the confidentiality protection system

17:37.505 --> 17:38.805
is supposed to protect.

17:39.945 --> 17:41.445
The reason there are

17:41.465 --> 17:44.605
so many more noisy measurements in 2020 than there are

17:44.605 --> 17:47.405
publication tables, 25 trillion to eight

17:47.405 --> 17:50.765
and a half billion approximately is the requirement

17:50.765 --> 17:51.885
to produce micro data.

17:52.865 --> 17:56.605
The requirement to produce micro data has some

17:57.155 --> 17:59.005
very distinct advantages.

17:59.825 --> 18:03.565
As soon as the top down algorithm finds a zero,

18:04.475 --> 18:08.285
that zero is preserved all the way down to the block.

18:10.495 --> 18:14.835
91% of all the statistics

18:15.495 --> 18:17.275
in the demographic

18:17.295 --> 18:19.875
and housing characteristics file are zeroes,

18:21.375 --> 18:25.035
and 91% of those zeroes

18:25.615 --> 18:28.195
are preserved by the top down algorithm.

18:29.265 --> 18:32.435
Neither of those, data points is confidential.

18:32.535 --> 18:33.595
You can confirm them

18:33.655 --> 18:36.035
by looking at the public use demonstration products

18:36.105 --> 18:37.555
that the Census Bureau published.

18:38.015 --> 18:41.475
So that means what the top down algorithm did is it made

18:41.475 --> 18:45.235
sure that the cells that actually have people in them

18:46.255 --> 18:47.835
got numbers in them.

18:48.185 --> 18:51.795
Even though the noisy measurements are unconstrained,

18:53.245 --> 18:55.825
the noisy measurement files are not sparse.

18:56.735 --> 18:58.825
Most of the entries are not zero.

18:59.365 --> 19:02.905
And the reason is that each statistic only got a really

19:03.155 --> 19:05.785
small proportion of the privacy loss budget.

19:06.205 --> 19:08.985
So the probability that a one or a two

19:09.005 --> 19:10.185
or a three was added

19:10.285 --> 19:12.385
to any particular cell is actually quite large.

19:12.685 --> 19:15.785
That's why the noisy measurements are not all zero

19:16.005 --> 19:17.105
or not mostly zero.

19:17.565 --> 19:20.625
So if I hand you the noisy measurements

19:21.285 --> 19:24.025
to do your redistricting from, you're gonna have

19:24.025 --> 19:25.185
to deal with negative numbers.

19:25.405 --> 19:27.545
You're gonna have to deal with numbers that don't add up

19:27.545 --> 19:29.265
to the, to the population totals

19:29.265 --> 19:30.825
of higher levels of geography.

19:31.685 --> 19:36.625
Um, most of the users of census data do not want to do this.

19:37.125 --> 19:38.225
And so,

19:38.285 --> 19:40.345
that's the reason why the top down

19:40.345 --> 19:41.585
algorithm was implemented.

19:42.245 --> 19:46.305
On the other hand, if you published the query workload,

19:46.525 --> 19:49.385
you published the actual tables, you pass them

19:49.385 --> 19:51.225
through a differentially private filter,

19:51.765 --> 19:53.465
and then you release them.

19:54.075 --> 19:58.545
There would only be for redistricting 1.5 billion for,

19:59.045 --> 20:01.905
for, DHC, about eight

20:01.905 --> 20:04.345
and a half billion total statistics.

20:04.725 --> 20:07.825
So that would be the same table format

20:08.045 --> 20:10.265
and the same organization

20:10.605 --> 20:12.145
as the historical published data.

20:12.445 --> 20:14.825
But each one of those numbers would be noisy

20:15.125 --> 20:17.585
and you would have to understand how to deal with it.

20:17.605 --> 20:18.665
That's the downside.

20:18.685 --> 20:21.905
The upside is all the privacy loss budget, all

20:21.925 --> 20:25.425
of the reduction in variance could be concentrated on the

20:25.425 --> 20:28.625
cells that you actually publish, not on the ones

20:28.625 --> 20:30.745
that you used in order to have to be able

20:30.745 --> 20:31.865
to publish for micro data.

20:32.405 --> 20:36.265
So the fundamental challenge in designing these systems is

20:36.505 --> 20:41.065
to compromise between, trying to make the data usable

20:41.685 --> 20:44.065
for, general audiences

20:44.605 --> 20:48.845
and trying to, protect the confidentiality in a way

20:48.875 --> 20:51.365
that, that is effective.

20:51.825 --> 20:53.765
And it isn't just differential privacy

20:54.075 --> 20:55.565
that addresses this issue.

20:55.635 --> 20:56.685
Systems in use

20:56.825 --> 20:59.245
by other national statistical offices have

20:59.245 --> 21:00.565
exactly the same issue.

21:01.305 --> 21:04.565
The basic way you protect confidentiality

21:04.745 --> 21:07.765
and tabular data, data tables is

21:07.765 --> 21:09.125
by adding noise to each cell.

21:09.825 --> 21:11.685
Um, the Census Bureau does

21:11.685 --> 21:13.365
that now in its census publications

21:13.745 --> 21:15.885
and other national offices do that.

21:16.075 --> 21:18.685
They didn't use differential privacy as the framework,

21:18.705 --> 21:20.365
but they did add noise to all the cells.

21:20.385 --> 21:22.525
And so they had to address what do you do

21:23.115 --> 21:24.205
with sparse tables?

21:24.285 --> 21:28.085
How do you handle the zeros that that, that are in spar?

21:28.165 --> 21:30.005
A sparse table means it's mostly zeros.

21:30.505 --> 21:32.925
How do you handle that? How do you restrict the users from

21:32.925 --> 21:36.005
asking queries that generate too many sparse tables?

21:36.005 --> 21:39.365
These are all the things that the top down algorithm handles

21:39.385 --> 21:42.765
algorithmically for the entire publication set.

21:43.105 --> 21:44.485
If you're not gonna do it that way,

21:44.485 --> 21:46.965
then you need a long list of ad hoc rules.

21:48.065 --> 21:49.805
And when you're all done applying them,

21:49.865 --> 21:51.805
you don't necessarily have a guarantee

21:51.805 --> 21:53.205
that you've got where you wanted to go.

21:54.555 --> 21:57.085
That said, some of these other systems seem to be easier

21:57.145 --> 22:00.285
to explain to the, user community,

22:00.645 --> 22:02.325
although they can be constrained in ways

22:02.325 --> 22:05.405
that would make them unusable in the us For example,

22:05.545 --> 22:07.605
the Office of National Statistics in the UK

22:08.905 --> 22:10.645
has a population lower bound.

22:10.705 --> 22:11.765
You can't have a cell

22:11.765 --> 22:13.405
that has fewer than a hundred people in it.

22:13.905 --> 22:15.205
If you apply that rule

22:15.665 --> 22:18.085
to the United States redistricting data, they're wiped out.

22:20.285 --> 22:21.685
They're not suitable for purpose.

22:22.065 --> 22:24.445
So if the statutory purpose is

22:24.445 --> 22:26.165
to redistrict all the legislatures,

22:26.385 --> 22:29.005
you gotta figure out a way to deliver cells

22:29.005 --> 22:30.685
that have smaller populations

22:30.785 --> 22:33.405
so you can get the granularity in the district size

22:33.405 --> 22:35.925
that's needed to create equal population,

22:36.545 --> 22:37.645
voting districts.

22:38.185 --> 22:40.725
And, and in the UK some of those data are used

22:40.865 --> 22:44.365
for redistricting, but the voting districts,

22:45.185 --> 22:48.285
are not based exclusively on the population counts.

22:49.745 --> 22:52.405
So, so the bottom line here is

22:53.425 --> 22:56.645
the query workload is the thing the users want to use.

22:57.345 --> 23:01.045
And the tension in protecting its confidentiality is between

23:01.665 --> 23:03.725
making it as usable as possible,

23:04.255 --> 23:07.725
which does introduce some biases in the statistics if

23:07.725 --> 23:08.965
they're tabulated from micro data.

23:09.145 --> 23:11.725
So micro data means the tabulations are consistent every

23:11.725 --> 23:14.165
which way they add up this way, they add up this way,

23:14.165 --> 23:15.285
they add up this way.

23:15.745 --> 23:17.485
So, so you could basically,

23:17.745 --> 23:21.005
you can use them the way you've traditionally used them, but

23:21.005 --> 23:23.205
because of the confidentiality protections,

23:23.735 --> 23:25.725
small cells have a small positive bias

23:25.785 --> 23:27.685
and large cells meaning highly

23:27.685 --> 23:29.405
populous, a small negative bias.

23:29.465 --> 23:30.685
And the same is true in systems

23:30.685 --> 23:32.085
that do output noise infusion

23:32.545 --> 23:36.725
and then make them, conform to a population total

23:36.725 --> 23:38.765
that hasn't been, infused with noise.

23:39.225 --> 23:44.125
That's a tension that is gonna continue to, to be

23:44.745 --> 23:46.405
in statistical agency.

23:46.505 --> 23:48.845
And it's also a tension that affects tech companies.

23:49.155 --> 23:52.365
Tech companies face the same kinds of problems when they try

23:52.365 --> 23:55.005
to let people use the, the data

23:55.105 --> 23:56.805
or when they use those data themselves.

23:57.895 --> 24:00.105
Some uses of the data are very disclosive

24:00.605 --> 24:03.545
and, they're also very useful.

24:03.665 --> 24:07.025
Like your advertising algorithms can be a lot more accurate

24:07.045 --> 24:09.065
if you let the people building them work

24:09.065 --> 24:10.185
with the confidential data

24:10.215 --> 24:11.865
that don't pass through a privacy filter.

24:12.005 --> 24:15.385
But some companies have elected not to allow that,

24:15.525 --> 24:17.305
and other companies still do allow that.

24:17.305 --> 24:19.825
And you can imagine the tension in the boardroom is the same

24:19.825 --> 24:23.185
as the tension that was faced in the Census Bureau on how

24:23.185 --> 24:27.305
to trade off accuracy for use ad revenue in this case,

24:27.525 --> 24:29.985
versus, protection of confidentiality.

24:30.645 --> 24:34.025
So this problem is, is with us, it's a,

24:34.055 --> 24:36.185
it's a 21st century problem

24:36.285 --> 24:39.825
and we're gonna have to, learn how to design systems

24:39.825 --> 24:40.705
that address it.

Overview of the 2020 US Census

00:00:05 - The US Decennial Census is a, very large data collection effort. The budget for the 2020 census was about $15 billion. The Constitution and statutory basis for, for that census requires it to produce data for some purposes. So it requires it to produce data for apportioning, the House of Representatives, which in the United States is the part of our, two Body Congress that is,
00:00:41 - So in principle, every member of the House of Representatives represents the same number of people. And, and so the apportionment of seats in the House of Representatives is, based on a formula. And that formula uses population counts, and they, by constitution have to come from the decennial census. So that's the constitutional basis. The next, the next activity that happens is those same data are used
00:01:14 - voting districts for the Congress, which is a national body, the voting districts for state legislatures, which are usually, two body bicameral like the federal government, except that both of those bodies are required to be one person, one vote. in addition to that, redistricting requires that the data, that the districts comply with the 1965 Voting Rights Act,
00:01:44 - about what constitutes a legal redistricting map. And so the data are used to support that, and that's where the reason of ethnicity, data come in. That's what they're used for. Now, that's also a statutory obligation. Finally, the, the statute says the data should be used to support the, estimation of population and population characteristics every year in what's known as the Population Estimates program at the Census Bureau.
00:02:15 - to produce more detailed demographic data on, on the population. And it's updated every year on the, the census data are supposed to be published in a way that supports their use in, in that those population estimates. Those population estimates are then used in many different formulas for apportioning resources. That's why they're, so critical On top of these, constitutional and statutory products, there are very detailed race and ethnicity data published.
00:02:48 - and household characteristics published. there have historically been, public use micro data that will also be true in 2020, but they're in a slightly different form. And then what's new for, 2020 is the, the Census Bureau has published the raw output from the disclosure avoidance system, which, no agency in the US or or in the world has ever done before. And I'll get into the details of all that work. But that disclosure avoidance system that you referenced in the start of the question,

2020 Census Confidentiality and Constraints Overview

00:03:20 - of the underlying census responses in a manner that allows all of these uses to be published. so it's a comprehensive confidentiality protection system. I'm going to do a little later how some of the constraints cause the way that system works to introduce known biases in certain statistics. But, basically it doesn't have anything to do with the differential privacy framework.
00:03:49 - that were put on the design of the system in order to make it meet these, constitutional and statutory requirements in order to protect the 2020 census confidentiality, which is a statutory requirement. It's in the Census Act for the United States. And to meet the use cases that I just described for publication, the senior executive staff at the Census Bureau, I was on that board, put five constraints
00:04:21 - So the first thing the system had to do is it could not affect the apportionment use case. So nothing about the confidentiality protection could affect the populations that are published for the 50 states of the District of Columbia, as they're used to apportion representatives in the House of Representatives. DC doesn't actually get any, but its number does get published. the District of Columbia doesn't get any. So, that's constraint one.
00:04:54 - the publication is set up for the purposes of redistricting has been pre-specified. It's pre-specified in a negotiation between the Census Bureau and a non-partisan group known as the National Conference of State Legislatures pre-specified in a way as to what this geographic area is that the data have to apply to. So that's called an atom. that geographic atom is something known
00:05:22 - And those census blocks are used to form all the voting districts of all of the legislative bodies, state, county, national, in the entire country. They do not have a population limit. They can be as small as zero, some are as big as a thousand people, but they're pre-specified. They've already been drawn, they were drawn ahead of taking the census, and so they can't be changed. When you find that one doesn't have any population, and it just is published with zero population.
00:05:57 - and ethnicity of the respondents to the census is pre-specified, because the minute those data are published, the redistricting offices in the States and the United States ingest them and start drawing redistricting plans. They have redistricting plans available the next day. So everything about those data has been pre-specified, and it was pre-specified in 2018. So those were constraints, they couldn't be changed.

2020 Census Confidentiality and Data Processing

00:06:30 - that the confidentiality protections could apply to was the final version of the 2020 census. It's known as the census edited file. So that's not just the record of what people said. It's also all of the edits that were used to make those responses usable data, filling in, missing data and other corrections, consistency corrections. The fourth constraint was that it had to deliver its output
00:07:01 - and implemented to publish the 2020 census known as the tabulation system. So the tabulation system had to ingest what are known as micro data, a record level image of each of the responses to the 2026, each household, and each person has a record, and that's what the tabulation system expected as input. Finally, the fifth constraint is the system couldn't delay the publication of the redistricting data. They had to be published on time.
00:07:29 - because of the pandemic, they were published four months late. But, that was because of the pandemic. The, the disclosure avoidance system, operated within its time budget, to allow that. So the way it works is it imposes a consistent differential privacy framework. That framework is known as zero concentrated differential privacy. It's an accounting framework for every statistic. There's precise measure of how much
00:08:04 - and you can add those, disclosure assessments up. That's the single biggest advantage of differentially private systems. They provide this accounting mechanism that lets you not guess at how disclosive the statistic is and not guess at how disclosive the whole package is. You have measures for both of those. And then it uses a mechanism, which is a specific way of adding random noise
00:08:34 - That mechanism is known as the discrete Gaussian mechanism, meaning it draws something that looks like in statistics of normal random variable, but normal random variables are continuous. You can't use continuous data in these systems that get compromises their implementation. You have to use integers. And so if you wanna use integer, then you Use a known as the discrete Gaussian mechanism. That is how the 2020 census disclosure avoidance system works.
00:09:05 - of the discrete Gaussian mechanism, and then it converts them to micro data by post-processing in an algorithm that has been specifically designed to make the micro data that gets tabulated as accurate as possible for the tabulation tables that eventually get published. So those things are all numerical calculations, they're estimated. And then, the trade-offs are assessed

Optimizing Census Data Through Hierarchical Levels

00:09:36 - that are under your control until you have tuned the system. And multiple demonstration products will produce to show how the system was tuned. So the top-down algorithm, the way it works is it starts by doing the best job it can with the whole country, and then it takes micro data for the whole country, which necessarily doesn't have any geographic identifiers on it because it's for the whole country. And it goes down to the next level in what's known
00:10:05 - And that next level is the state. So taking the best you can do to produce micro data at the national level, as given you then produce all the state level tables. So that's, 50 states, the District of Columbia and the Commonwealth of Puerto Rico, all those states, all those tables are constrained to aggregate to the national table, but they are otherwise the best you can do state by state. Once you have that done, you pass to the next level of the geographic hierarchy, which is known as counties
00:10:38 - I'm abstracting from some details here. So people familiar with the system will recognize that, but it's the best way to describe it. At the county level, you take the state level as given, and then you produce micro data that gives the best county tables that are consistent with the state, and those are also consistent with the nation. Then you pass to the next level of geography. there are, depending on which system we're talking about, the levels now get more complicated. But basically there's a tabulation area known
00:11:10 - a tabulation area known as a census block group, which is the next area. And then finally you get to those atoms that I described before, which are the census blocks. And in the 2020 census, there are 8 million of those census blocks. So at that point, you're optimizing 8 million table sets so that they aggregate to the about 250,000, block group table sets, which then aggregate to the about 75,000 tracked aggregates.
00:11:41 - so they can then be sent as a single record that has all of those geographic identifiers in it to the tabulation system, which then produce the tables according to the same way that users are used to using them from previous censuses. What the Census Bureau has labeled noisy measurement files are the raw output from the disclosure avoidance system from the top-down algorithm. And what that means is that the top-down algorithm operates in two steps.
00:12:16 - and then the second step does the optimization that I just described to produce the micro data that are subsequently tabulated. So the noisy measurements themselves constitute every question that the designers of the system determined needed to be answered. Those are called the query strategy. They are put directly to the confidential data,

Understanding Noisy Measurements in Census Data

00:12:47 - You add the noise from the, discreet gian mechanism. The confidential number plus the noise is a noisy measurement. And there are two sets of noisy measurement files. They're the noisy measurements that were taken to produce the redistricting data, and then the noisy measurements that were taken to produce what's called the demographic and housing characteristics, DHC data,
00:13:15 - that I described, earlier. So to produce the redistricting data, 16 billion, that's a billion with a B. Noisy measurements are taken. The query strategy in order to tabulate 1.5 billion statistics. The actual redistricting data are about 1.5 billion statistics. The reason there are so many more noisy measurements taken,
00:13:49 - that requirement to convert the noisy measurements to micro data. That requirement is what, means. You're going to ask many more queries than you're going to publish the answers to. Right? For the demographic and housing characteristics, there are 25 trillion with a t noisy measurements, even though there are only about 8 billion or eight
00:14:18 - that those answers are approximate is it's very difficult to count the number of linearly independent statistics the way the Census Bureau published tables. So those are, approximations that are, that are, certainly the right range. The reason that there's so many more noisy measurements for DHC than there are for the redistricting is that there's a lot of very detailed queries that have to be answered in order to publish the kinds of tables
00:14:47 - and housing characteristics, which not only tell you a person's sex and, that person's, race, they tell you, their age and a variety of different, categories. They tell you things about the composition of the household. They tell you some things about the structure of the housing unit, that requires taking what we call a detailed query that has very, very high dimension. And that's why there are 25 trillion
00:15:19 - There are so many of those noisy measurements. They couldn't be handed to the census bureau's dissemination system to publish that didn't fit on the entire server farm. So separate arrangements were made with, with platforms that had access to, cloud computing environments, and they were to, they were deployed to cloud computing environments in order to allow them to be analyzed. so those noisy measurements have no post processing done to them. They're exactly the confidential number, plus the noise.

Understanding Noisy Measurements in Census Data

00:15:54 - are requesting the noisy measurements is that, there's a lot more information in the noisy measurements than there are in the published tables, because the noisy measurements had to take many more measurements than eventually got tabulated. So that's really the same thing as asking for micro data. It's just in a very different format. The 25 trillion noisy measurements from the DHC,
00:16:25 - You can ask questions of those data at any level of geography that is in the, the top down algorithm for any detail. That's in what's called the detailed query. So lots of things that weren't in published tables have an answer in the noisy measurements. It's a noisy answer, but it's already been passed through the confidentiality protection system. The privacy loss has already been expended. And so the researchers can use that, to, to, to ask questions that they're not able to ask
00:16:56 - So that's what the noisy measurements are. It was a major innovation to publish them. There will also be a public use, file that looks like a public use micro data file, but it is actually the, the post-process micro data. It's the, the, the record that comes out of the top down algorithm, not the record that went into the top down algorithm. So that's what the noisy measurements are, the difference between the noisy measurement files and the query workload output.
00:17:27 - The query workload output is the tables that actually get published in principle, those are the tables that the confidentiality protection system is supposed to protect. The reason there are so many more noisy measurements in 2020 than there are publication tables, 25 trillion to eight and a half billion approximately is the requirement to produce micro data. The requirement to produce micro data has some
00:17:59 - As soon as the top down algorithm finds a zero, that zero is preserved all the way down to the block. 91% of all the statistics in the demographic and housing characteristics file are zeroes, and 91% of those zeroes are preserved by the top down algorithm.
00:18:32 - You can confirm them by looking at the public use demonstration products that the Census Bureau published. So that means what the top down algorithm did is it made sure that the cells that actually have people in them got numbers in them. Even though the noisy measurements are unconstrained, the noisy measurement files are not sparse. Most of the entries are not zero.

Challenges in Census Data Privacy Management

00:19:03 - small proportion of the privacy loss budget. So the probability that a one or a two or a three was added to any particular cell is actually quite large. That's why the noisy measurements are not all zero or not mostly zero. So if I hand you the noisy measurements to do your redistricting from, you're gonna have to deal with negative numbers. You're gonna have to deal with numbers that don't add up to the, to the population totals of higher levels of geography.
00:19:37 - And so, that's the reason why the top down algorithm was implemented. On the other hand, if you published the query workload, you published the actual tables, you pass them through a differentially private filter, and then you release them. There would only be for redistricting 1.5 billion for, for, DHC, about eight
00:20:04 - So that would be the same table format and the same organization as the historical published data. But each one of those numbers would be noisy and you would have to understand how to deal with it. That's the downside. The upside is all the privacy loss budget, all of the reduction in variance could be concentrated on the cells that you actually publish, not on the ones that you used in order to have to be able to publish for micro data.
00:20:36 - to compromise between, trying to make the data usable for, general audiences and trying to, protect the confidentiality in a way that, that is effective. And it isn't just differential privacy that addresses this issue. Systems in use by other national statistical offices have exactly the same issue. The basic way you protect confidentiality
00:21:07 - by adding noise to each cell. the Census Bureau does that now in its census publications and other national offices do that. They didn't use differential privacy as the framework, but they did add noise to all the cells. And so they had to address what do you do with sparse tables? How do you handle the zeros that that, that are in spar? A sparse table means it's mostly zeros. How do you handle that? How do you restrict the users from asking queries that generate too many sparse tables?
00:21:39 - algorithmically for the entire publication set. If you're not gonna do it that way, then you need a long list of ad hoc rules. And when you're all done applying them, you don't necessarily have a guarantee that you've got where you wanted to go. That said, some of these other systems seem to be easier to explain to the, user community, although they can be constrained in ways that would make them unusable in the us For example, the Office of National Statistics in the UK

Balancing Confidentiality and Redistricting Data Use

00:22:10 - You can't have a cell that has fewer than a hundred people in it. If you apply that rule to the United States redistricting data, they're wiped out. They're not suitable for purpose. So if the statutory purpose is to redistrict all the legislatures, you gotta figure out a way to deliver cells that have smaller populations so you can get the granularity in the district size that's needed to create equal population, voting districts. And, and in the UK some of those data are used
00:22:45 - are not based exclusively on the population counts. So, so the bottom line here is the query workload is the thing the users want to use. And the tension in protecting its confidentiality is between making it as usable as possible, which does introduce some biases in the statistics if they're tabulated from micro data. So micro data means the tabulations are consistent every
00:23:14 - they add up this way. So, so you could basically, you can use them the way you've traditionally used them, but because of the confidentiality protections, small cells have a small positive bias and large cells meaning highly populous, a small negative bias. And the same is true in systems that do output noise infusion and then make them, conform to a population total that hasn't been, infused with noise. That's a tension that is gonna continue to, to be
00:23:46 - And it's also a tension that affects tech companies. Tech companies face the same kinds of problems when they try to let people use the, the data or when they use those data themselves. Some uses of the data are very disclosive and, they're also very useful. Like your advertising algorithms can be a lot more accurate if you let the people building them work with the confidential data that don't pass through a privacy filter. But some companies have elected not to allow that,
00:24:39 - And you can imagine the tension in the boardroom is the same as the tension that was faced in the Census Bureau on how to trade off accuracy for use ad revenue in this case, versus, protection of confidentiality. So this problem is, is with us, it's a, it's a 21st century problem and we're gonna have to, learn how to design systems that address it.

Faculti AI can make mistakes

Research

Policy

Analysis

Data

Originals

Explainers

Podcasts

Faculti

New on Faculti

Idea Explorer

Definition

News Context

Critical Questions and Answers

Research Summaries

Noisy Measurements Are Important, the Design of Census Products Is Much More Important

Media Literacy Toolkit

Critical Questions to Consider ▼

Bias and Perspective Awareness ▼

Further Reading ▼