Meta’s $17 Billion Settlement is a Bad Deal for Teens and All Social Media Users

Meta’s settlement with 52 state attorneys general is a bad deal for all internet users, and especially for teens. That’s what we said the day the settlement was announced.

In this post, we go through the Settlement’s provisions in detail and explain why that is so, including how:

• The Settlement embeds age assurance technology and age-gates into Meta’s social media products and requires all users—minors and adults—to undergo a rights-threatening age estimation process
• The Settlement places severe restrictions on Teens that can only be modified by the Teens’ parents and only then in exchange for giving their parents a ton of information about their online community and usage;
• The Settlement seems to empower the attorneys general to enforce Meta’s content restriction on “age inappropriate content,” categories that Meta itself has had trouble administering without excluding information about sexuality, sexual and reproductive healthcare, and abortion medication;
• The Settlement actually requires Meta to collect, analyze, and retain more information about its Teen users—when the pressure should have been on Meta to diminish its surveillance capitalism.

Note: A chunk of the settlement addresses unauthorized users under age 13, and Meta’s obligations to comply with the Children’s Online Privacy Protection Act. Meta policy has banned users under 13 since the company opened to the public in 2006. Aside from the age assurance frameworks that support both those and the other parts of the Settlement, the under-13 provisions are not addressed in this post. Those provisions essentially require Meta to detect and delete all under-13 accounts.

Further note: All U.S. states are parties to the Settlement except Florida, New Mexico, and Texas. The Settlement includes D.C., American Samoa, Guam, Northern Mariana Islands, and Puerto Rico.

Age Gates Reinforced By Age Estimation Technology

In the Settlement, Meta agrees to age-gate Instagram and Facebook, thus making age gates a legal mandate. And Further, Meta will now enforce these age gates with “age assurance” technology, ditching its previous practice where the person signing up for the services self-attests to their birthdate. This concession firmly embeds deeply flawed age estimation technology into the online experience of millions of people around the world. First and foremost, the age verification setup seriously threatens online anonymity and privacy for everyone, as we’ve said before. The Technology also just adds a layer of creepiness into the user of any service. In the Settlement, Meta pledges to, within one year, apply one or more age assurance methods to each Instagram or Facebook user in the states and territories that joined the Settlement. [P. 10, §II.A.1]

1. Age Assurance Framework. Within one (1) year of the Effective Date, Meta will adopt an age assurance framework (“Age Assurance Framework”), wherein it will apply one or more age assurance methods developed by a third party and licensed to customers (“Commercially Available Age Assurance Methods”) orage assurance methods developed by Meta (“Proprietary Age Assurance Methods”) (collectively, “Age Assurance Methods”) to each Meta SMP user in the Settling States. For the purposes of this Section II.A, an age assurance method developed or acquired by Meta that uses the same or functionally identical technology and methodology to a Commercially Available Age Assurance Method shall be treated as a Commercially Available Age Assurance Method. The Age Assurance Framework must include Age Assurance Methods to evaluate whether a Meta SMP user is a Teen User or U13, as described in Section II.A.6. New users of Meta SMPs who have not yet had their age assessed by an Age Assurance Method pursuant to Meta’s Age Assurance Framework shall receive the Default Protections pursuant to Section II.A.10 of this Agreement.

Those methods might include both commercially available products, as well as proprietary age estimation process Meta might have or develop. Meta also pledges to consider age signals from Google and Apple operating systems and app stores. [§II.A.5] Meta has previously advocated for age assurance requirements to fall on Google and Apple rather than on individual services.

This age assessment essentially dumps users into one of three age-range buckets: 18+, 13-17, and under-13. Users under 13 have long been barred from Meta products, but this Settlement creates new obligations to search for and detect users who may have said they were older.

For those estimated to be over-18, the Settlement guarantees no direct benefit to you: no privacy protections, no greater user controls for your own accounts, no dent in Meta’s surveillance capitalism.

Those estimated to be 13-17 years old will be limited to Teen User accounts.

Those estimated to be under-13 will lose their accounts altogether.

Those who open new accounts will have two weeks to submit to age estimation, and if they decline to do so, Meta is now required to treat them as a Teen User by default, even if they self-identify as being 18 and older. [P. 18, §II.A.10.b]

(b) Fourteen (14) days or more after creating a Meta SMP account, Meta SMP users who have not yet had their age assessed by an Age Assurance Method pursuant to the Age Assurance Framework shall be treated as Teen Users for the purposes of this Agreement regardless of their stated age, except that Meta SMP users with a stated age of 18 years old or older shall receive the protections described in Section II.A.10.a.ii.

What about people with existing accounts, who are well past that two-week period to submit to age estimation? Will they also be defaulted to Teen User status if they decline age estimation? It seems so—the AGs would likely not have accepted a settlement that did not require Meta to take action against existing teen users who choose to forgo the age assurance process. Perhaps Meta will use its existing store of information about its current users as a type of permitted proprietary age assurance process? Thus, perhaps, an adult user whose Facebook account is itself older than 18 will be assessed as being over-18? Or a user who is identified as the spouse of a user who has been age-assured? But Meta can only rely on a proprietary process if it meets the accuracy standards set out in the Settlement Agreement.

How accurate does the age assurance process need to be?

The Settlement sets maximum false positive for both commercially available and proprietary age assurance methods [Pages 12-13, §II.A.6]. Within two years, each shall be no more 10% for ages 16-17 and 3% for ages 13-15. 

6. Age Assurance Standards.
(a) U18 False Positive Rate Thresholds.
(i) Any Commercially Available Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall
meet or fall below the following U18 False Positive Rates excluding method circumvention within one year of the Effective Date: 10% for minors aged 16-17 and 3% for minors aged 13-15.
(ii) Any Proprietary Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall meet or fall below the following U18 False Positive Rates excluding method circumvention within 1 and 2 years of the Effective Date: 
(A) Year 1: Within one year of the Effective Date: 14% for minors aged 16-17 and 7% for minors aged 13-15.
(B) Year 2: Within two years of the Effective Date, 10% for minors aged 16-17 and 5% for minors aged 13-15.

Notably, there is no limit indicated in the error rate for false negatives, when the process wrongly identifies an adult as being under 18. The Settlement only requires Meta to maintain an appeal process for users whose age range is wrongly assessed. [P. 17, §II.A.9]

9. Appeals Process. Users claiming to have been mis-identified as minors must be offered a Clear and Conspicuous means to appeal the decision. Decisions on all user appeals must be made in a timely manner and communicated to the user along with a basis for the decision.

The Settlement generally shows little concern for those falsely placed in its Teen User category.

Meta must also employ measures to discourage age estimation circumvention, including placing limits on the number of attempts any user might make. [P. 16, §II.A.7] As part of this, Meta agrees to proactively monitor adult accounts to determine whether a user needs to undergo additional age estimation. [P. 16, §II.A.7.c] This is just one of the ways the Settlement embeds Meta’s active surveillance of its users for the next ten years (see below for more).

(c) Incorporating a proactive monitoring system that requires users to undergo an additional Age Assurance Method where a user is determined, including based on their conduct on Meta SMPs, to have likely circumvented the Age Assurance Method and is: (A) likely a Teen User after having been previously assessed as 18 or older; or (B) likely U13 after having been previously assessed as 13 or older. Users Meta determines are likely Teen Users after having been previously assessed as at least eighteen may choose not to undergo additional age assurance but then will be treated as Teen Users; and 

Any age assurance process Meta uses must be tested annually.

Data minimization

The Settlement does have data minimization requirements for the data collected during the age assurance process. [§II.A.8] But there are numerous holes. The Settlement requires that all information obtained and retained as part of the age assurance processes thereafter be “immediately enqueued for deletion, after which it shall be deleted after a reasonable period of time.” But the Settlement defines a category of “Retainable Data” that may be retained for 90 days. This includes “metadata about the age assurance method used by the user information … where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes.” The Settlement requires at that all data collected by Meta or its vendor be stored according to industry-standard data security measures,” a standard that unfortunately does not eliminate the risk of a data breach.

8. Data minimization and security.
(a) Except as set forth herein, all data collected by Meta from users of Meta SMPs in the Settling States for the sole purpose of conducting age assurance, all data maintained from known U13s, and all data collected by a vendor for use in a Commercially Available Age Assurance Method shall be held for the minimum period required to determine a user’s age status and thereafter immediately enqueued for deletion, after which it shall be deleted in a reasonable period of time. Meta may retain (1) U13 data only to the extent required for purposes of developing, training, testing, and measuring the performance of the U13 Age Model (“U13 Data”), provided that any U13 data that constitutes Personally Identifiable Information as defined by 16 C.F.R. Part 312.2 will be protected using Meta’s highest data privacy and security standards, and (2) metadata about the age assurance method used by the user information (“Retainable Data”) only where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes. For the avoidance of doubt, U13 Data cannot be used for purposes such as ads targeting and delivery, marketing, or algorithmic optimization efforts. Any U13 Data and Retainable Data shall be maintained at the coarsest viable granularity and cannot be used for any other purpose unless legally required. Any Retainable Data that is no longer required for the purposes set forth herein shall be deleted within 90 days. The terms above shall not pertain to the user’s stated date of birth, stated age, nor the outcome of the Age Assurance Method (e.g., “teen or adult” classification).
(b) Data collected by Meta or a vendor, or transmitted by a vendor, must be collected and stored using industry-standard data security measures and as required by law, including encryption in transit and at rest.
(c) The Parties agree to discuss in good faith potential modifications to this provision as necessary to permit Meta to improve the efficacy of its Age Assurance Framework while preserving the principles of data minimization and security set forth in this Section.

Restrictions For Teens (And Anyone Who Opts Out of Age-Gating)

Teen User Accounts are subject to time, feature, and content restrictions. These will be applied to these estimated to be 13-17 as well as any new user who declines to submit to the age assurance process and to existing users who decline to submit to age estimation and for whom Meta cannot ascertain that they are older than 17.

Time restrictions

Teen Users will be subject to the following time restrictions (§II.B). These measures seem to assume that most of teen’s social media use is frivolous and unserious (not that those are in and of themselves bad; the right to play is among young people’s human rights), ignoring the fact that teens use social media for school and personal research, conducting activism campaigns, and other endeavors that might naturally not fit within these time limits:

• Night Access Mode – no access (except messaging) to Instagram and Facebook from Midnight to 6 AM, and no push notifications from 10 PM – 7 AM.
• School mode – no push notifications from 8 AM – 3 PM Monday-Friday from Aug 15-June 15.
• Daily cumulative time limit of 2 hours per day across Instagram and Facebook, resetting at midnight, excluding video and audio content at least 22 minutes long, absent artificial prolongation, defined by the Settlement as “longform content.”
• “Productive pauses and notices” designed to “reduce or prevent excessive, mindless, or unintended teen usage.” This means that a teen’s usage will be monetarily paused after 60 and 90 minutes of daily cumulative use with notices sent every 15 minutes of continuous use. According to the Settlement, these productive pauses and notices will look like this:

To be clear, the ability to set time limits, blackout times, and scheduled pauses are all useful features that should be available and easy to implement for users of all ages. Such tools would have allowed teens, and all users, the ability to design their own safe experience, customized to their own needs, online. Such users controls would have recognized that teens have human rights, agency, and autonomy.

But that’s not what these restrictions are. They are not tools that give the teen users control. Rather, they are imposed, top-down, on teens and anyone else who declines to submit to Meta’s age assurance process.

Feature restrictions (§II.C-D)

Within four months of the effective date of the Settlement, Meta must offer teens an option for a non-personalized feed, which is defined as a feed of chronologically ordered posts from friends and follows. Teens will also be able to disable autoplay as part of an “optional protective settings” package. Each of these settings must be “viewable within three user gestures and clearly labeled, easy to notice, viewable without scrolling, and discoverable in an intuitive location within” the service.

Again, these would be useful user controls that should be offered to users of all ages.

By default, teens will not see the number of likes or other reactions to their posts.

Teens will also not have access to what the Settlement calls “Cosmetic Procedure Filters,” that is, “any digital filter or augmented reality effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme make-up techniques. 

X. “Cosmetic Procedure Filter” shall mean any digital filter or augmented reality (AR) effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme makeup techniques. For the avoidance of doubt, “Cosmetic Procedure Filter” does not include: (1) Fantasy/Character Effects: Filters that change a user’s facial structure for the purpose of turning the user into a non-human or fictional character (e.g., an elf or cartoon) or an animal (e.g., a dragon or puppy); (2) Makeup/Smoothing Effects: Filters that smooth skin or alter appearance in ways that can be achieved by ordinary makeup techniques without altering the appearance of underlying physical facial structure or meaningfully changing skin tone; or (3) Parody and Exaggeration Effects: Effects that entertain users by distorting their appearance through parody, satire, or exaggerated forms (e.g., extreme visual distortions outside the scope of normal cosmetic procedures). To help operationalize this definition, the Settling States will provide Meta illustrative examples and guidance of AR effects that are Cosmetic Procedure Filters and AR effects that are not Cosmetic Procedure Filters in a letter to be sent within two (2) months of the Effective Date.

Meta has already had rules about cosmetic effects directed at teens since 2019. But the Settlement will give the states a major role in helping Meta identify what features are and are not Cosmetic Procedure Filters. 

Content restrictions (P.1, §II.E, as defined by §I.C, E, F)

For content, Meta is basically pledging to continue its existing practices limiting Teen Users to age-appropriate content and accounts, to default Teen Users to age-appropriate experiences. This includes limiting access to accounts that “regularly share content that is inappropriate for teens” such as content from the following Meta community standards categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders.

C. “Age Appropriate Experiences” refers to content that is appropriate for Teen Users based on input from experts, parents, and teens. For the purposes of this Agreement, Age Appropriate Experiences shall mean content captured in Meta’s applicable Ages 13+ content setting, which is tied to policies inspired by movie ratings for ages 13+ and parent feedback.
D. “Age Assurance Methods” shall have the meaning set forth in Section II.
E. “Age Inappropriate Accounts” refers to accounts that regularly share content that is inappropriate for teens or that have account information that otherwise suggests the account is inappropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Accounts shall mean accounts that: (1) regularly share Age Inappropriate Content in the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders, or (2) have account names or profile photos or bios that suggest the account is otherwise inappropriate for minors, based on Meta’s policies for the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders.
F. “Age Inappropriate Content” refers to content that is generally perceived by U.S. parents, youth experts, and teens as not being appropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Content shall mean content prohibited by Meta’s Community Standards concerning bullying and harassment; nudity and sexual activity; child sexual exploitation, abuse, and nudity; sexually explicit language; suicide, self-harm and eating disorders; graphic violence and incitements to violence; gambling; and restricted substances or goods (including illegal drug use), as well as policies specifically focused on protections for Teen Users, including those regarding high-risk viral challenges and risky stunts.

The issue here is that some of these categories are problematic. For example, the Restricted Goods & Services standard has been used by Meta to justify removing information about abortion medication, as we detailed in our Stop Censoring Abortion campaign, and in our comment to the Meta Oversight Board. And under the Adult Nudity & Sexual Activity, Meta blocks teens from “real world art of visible genitalia … where the nudity is the focus of the image” and has a history of applying the standard inconsistently, including with respect to representations of indigenous women, breast cancer awareness posts, and posts about testicular and breast self-exams, educational posts about ovulation. And it has disproportionately appliedto gay and lesbian content in as compared to straight content.  

And even more worrisome, even though this is just Meta continuing its existing practices, the Settlement empowers the states to enforce its provisions. [P. 40, §IV.C.1.i; §VII.C] That means that over the next ten years, the duration of the Settlement, Meta will face the threat that a state attorney general will pursue legal action against it because it disagrees with how Meta interprets these categories of community standards, and pressure Meta to eliminate Teen User access to posts about sexuality and reproductive and sexual health. And Meta will now lack the hard-earned First Amendment defenses to make its own curatorial decisions. 

C. Notwithstanding anything in Sections VIII.A-B above, a Settling State may take any action, including but not limited to legal action to enforce compliance with the Agreement, without delay if the Settling State believes that a threat to the health or safety of the public requires immediate action.

The Parental Supervision Tradeoff 

All of these Teen User restrictions can be modified – but only if the Teen User enrolls in the Parental Supervision that links their account to a parent’s or guardian’s account. Once their accounts are linked, Parents can modify the Teen User settings to make them less restrictive (they need the teen’s permission if they want to make them more restrictive).

And Parental Supervision comes with a huge privacy tradeoff. In exchange for designating someone as their Parent, the Parent gets a lot of information about the Teen’s use: the usernames of all of the teen’s connections, reports on how much time the Teen User spends on a Meta service, the time spent watching longform content, usernames of all those messaging with Teen User, and any evidence Meta has about suspected secondary accounts. The Supervising Parent also gets notices of the teen’s repeated searches related to suicide, self-harm and eating disorders. [P. 28, §II.G] 

Parental Supervision
1. Meta agrees to provide Supervising Parents with information concerning the amount of time their Teen User is spending on Meta SMPs, including separately for time on the Meta SMP, time using the Meta SMP’s messaging features, time viewing Longform Content (to the extent excluded from the calculation of daily limits pursuant to Section II.B.3.a.i), and the usernames of the Teen User’s social connections and individuals messaging the Teen User, and usernames of any user reported by a Teen User. In addition, Instagram will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders, and Facebook will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders.
2. For Teen Users, Supervising Parents will be notified on a daily basis when the Teen User communicates directly with each adult user for the first time with the Teen User and shall provide a link to the adult user’s profile to provide information such as their stated hometown or city or mutual connections, to the extent the information is publicly available.
3. Upon a Teen User enrolling Parental Supervision, Meta SMPs shall prompt the Supervising Parent to review their Teen User’s settings to consider making updates to support how their Teen User spends their time on Meta SMPs. Meta shall not preselect, recommend, or encourage settings to Supervising Parents that are less restrictive than the default settings in this Agreement.
4. Meta SMPs will periodically suggest checkups for Supervising Parents to do with their Teen Users to evaluate their settings and usage.
5. Meta SMPs agree to continue to take steps designed to encourage enrollment in their Parental Supervision tools.
6. If a Teen User is enrolled in Parental Supervision, Meta shall notify the Supervising Parent if the Teen User creates or explicitly links a new secondary Meta SMP account within their Accounts Center or has been linked to a Soft Matched Account through Meta SMPs, including a link to the secondary account’s profile. Meta SMPs shall automatically apply the Supervising Parent’s approved time management settings in Section II.B and content restrictions to all explicitly linked Meta SMP accounts and all Supervised Accounts for a Teen User belonging to the same Meta SMP.

Unfortunately, Meta doesn’t have a great way to learn whether there exists a genuine parent-Teen User relationship. But it will try to get it right. [§II.G.8] 

This may be workable for young people with healthy and safe relationships with their parent or guardian. But obviously not good at all for a Teen User lacking such a safe relationship. 

More Surveillance, Not Less

Moreover, rather than pushing Meta away from the collection, analysis, and retention of user data, the Settlement requires Meta to do more of it for the next ten years. Several provisions of the Settlement require Meta to carefully track its users use of Instagram and Facebook for the purposes of determining whether its age assurance framework and Teen User restrictions are working as intended.  

For example: 

• Meta promises to review detected U13 users’ friend networks to identify other possible U13 users. [§II.A.6.(b).(i).(D)] 

• Meta will incorporate a “proactive monitoring system” to identify possible Teen Users among those who were initially determined to be older than 17 by its age assurance process. [§II.7.(c)] 

• Meta pledges to utilize and improve its existing “soft matching models” that track signals such as device IDs, phone numbers, and email addresses” to identify duplicate accounts. [§II.B.6.(a)] 

• Meta will continue to monitor Teen Users’ activity to “regularly evaluate the prevalence of Teen User exposure to Age Inappropriate Content, and Teen User exposure to experiences that are not Age Appropriate Experiences,” [§II.E.3] and also to “regularly evaluate the prevalence of Teen User exposure to Harmful Experiences on Meta SMPs.” [§II.F.3] 

• Monitor Teen Users’ messaging to identify message threads with potentially suspicious accounts. [§II.F.4] 

• The Settlement also requires Meta to actively monitor use of its services so that it can supply data to the Independent Auditor that will now monitor its compliance with the Settlement. “The Independent Auditor is entitled to access the non-privileged information, personnel, systems, and records that are reasonably relevant and sufficient to evaluate Meta’s implementation of the Injunctive Relief Terms, including, but not limited to, access to raw data; aggregated data; information; internal documents and communications” plus information from its age assurance processes, data regarding Teen User responses to the Productive Pauses, data regarding its models for soft matching of secondary accounts, data regarding the prevalence of Teen User exposure to Age Inappropriate Content and Harmful Experiences, and to experiences that are not Age Appropriate Experiences. [§III.E] 

Moreover, one of the chief threats of Meta’ surveillance is the honeypot of data it creates that may be accessible to governments for law enforcement and other investigations. Nowhere in the Settlement do the 52 attorneys general pledge to not try to access all of the data the Settlement requires Meta to collect and retain. 

Meta Has To Pay The States — Establishing Norms Beyond Meta

The Settlement also includes annual payments from Meta to the states, apparently proportionate to the size of each state’s teen user base. Over the ten year-life of the Settlement, these annual payments will total over $11 billion plus the prospect of an additional $5 billion—if Meta competitors adopt the same measures. 

This quirk of the Settlement incentivizes the States to pursue similar age assurance processes and at-least-as-restrictive teen user measures for Meta’s chief existing competitors for teen use, YouTube, TikTok, and Snap, and for any new service that may gain widespread teen use over the life of the Settlement Agreement. If the states are able to get Meta’s competitors to adopt the same measures, then the states will get an additional $5 billion in annual payments. That’s quite the incentive for the states to pursue litigation and regulatory measures against those companies. All of this will further entrench age assurance and age-gating as the norm across online services. 

1. In the event the Contingent Monetary Payment Trigger has occurred in a Settling State, Meta shall be obligated to pay to such Settling State ten equal installments in the amount as set out in Exhibit B (each, a “Contingency Installment Payment”).

2. Following the date the Contingent Monetary Payment Trigger has occurred, the Contingency Installment Payments shall be made to the Settling State on January 15 in each subsequent calendar year of the Agreement Term as follows:

(a) At the next scheduled payment date, the Settling State shall be paid the Contingency Installment Payment for that payment date and all prior payment dates.
(b) For each of the remaining payment dates, the Settling State will be paid the Contingency Installment Payment for that payment date.

3. If a Settling State fails to achieve the Contingent Monetary Payment Trigger during the Agreement Term, the Contingency Installment Payments shall be permanently forfeited by such Settling State and retained by Meta. For the avoidance of doubt, no Settling State will have an obligation to repay the Contingency Payment

The Settlement is thus a bad deal for all users of Facebook and Instagram. It normalizes age gating and age assurance for millions of internet users. It denies teens the tools to create their own safe experiences online and places their social media experience firmly under the control of either Meta or their parents. And rather than addressing Meta’s collection, analysis, and retention of data about teens’ use of Instagram and Facebook, it binds Meta to continued surveillance.