Artificial Intelligence: A Conceptual Learner's Series

Part 10: Ethics, safety and the future

Author: Christopher DanosAI by Claude

As AI systems make consequential decisions, the questions shift from "can we build it?" to "should we, and how do we make it fair, safe and accountable?" Many of these questions are actively debated, so this part lays out the main positions rather than settling them.

Concept 1: Fairness and bias

In one line: AI systems can treat groups of people unequally, usually by learning patterns from historical data that reflect past inequalities, and even defining "fair" turns out to involve hard trade-offs.

Example: A company trains a résumé-screening model on ten years of its own hiring decisions. Historically, it hired mostly men for engineering roles. The model learns that signals associated with women, such as attending certain colleges or listing certain activities, predict "not hired." Nobody programmed it to discriminate, and gender isn't even an input. It learned the old pattern from proxies and repeated it at scale.

Formally:

Bias in this sense means systematic, unjustified differences in outcomes or errors between groups.
Sources include historical bias (the past was unequal), representation bias (some groups are underrepresented in data), measurement bias (labels or features measure different groups differently) and proxy variables (features that stand in for protected attributes).
Competing fairness criteria include demographic parity (equal rates of positive outcomes across groups), equalized odds (equal error rates across groups) and calibration (a score means the same probability for every group).
Researchers have shown that when base rates differ between groups, several of these criteria cannot all be satisfied at once, so choosing among them is a values decision, not only a technical one.

Connections: Biased labels are a learning problem (Part 6, Concept 1); which errors matter most is a utility question (Part 5, Concept 1).

Watch out:

Removing the sensitive attribute isn't enough. Proxies like zip code can carry the same information.
There's no neutral default. Every fairness definition favors some trade-offs over others, and reasonable people disagree about which to prioritize.

Concept 2: Privacy and surveillance

In one line: AI makes it far cheaper to collect, combine and analyze personal data, which creates real benefits and real risks to people's privacy.

Example: A fitness app publishes an anonymized heat map of where users run, to show popular routes. Nobody's name appears. But in remote areas, the only runners are soldiers at a base, and their routes trace its layout. Separately, combining a few "harmless" facts, such as zip code, birth date and gender, can single out many individuals in a supposedly anonymous dataset.

Formally:

Re-identification: linking anonymized records back to individuals by combining datasets.
Inference risks: models can predict sensitive traits (health conditions, beliefs) people never disclosed.
Differential privacy adds carefully calibrated noise so that results reveal patterns about groups while limiting what can be learned about any individual.
Federated learning trains models across many devices without collecting raw data centrally.
Data minimization: collect and keep only what a task needs.
Uses such as facial recognition in public spaces raise debates that weigh safety, convenience, civil liberties and consent.

Connections: Generative models (Part 7, Concept 5) can memorize and reproduce training data, which links privacy to model design.

Watch out:

"Anonymized" often isn't. Treat removing names as the start of privacy protection, not the end.
Privacy protections cost accuracy. Techniques like differential privacy involve a trade-off between protection and usefulness.

Concept 3: Transparency, explanation and accountability

In one line: When an AI system affects someone's life, people want to know why it decided what it did and who is responsible when it gets things wrong.

Example: A loan applicant is rejected by an automated system. She asks why. If the system is a small decision tree, the bank can say: debt-to-income ratio above the threshold. If it's a large neural network, the honest answer may be harder to give. And if the decision was wrong, who's responsible: the bank, the vendor who built the model, or the team that assembled the training data?

Formally:

Interpretable models (small trees, linear models) can be understood directly. Post-hoc explanations approximate why a complex model made a decision, for example by measuring which inputs most influenced it.
Feature attribution methods estimate each input's contribution to a prediction. Counterfactual explanations say what would have had to change for a different outcome ("approved if debt were $5,000 lower").
Mechanistic interpretability studies the internal computations of neural networks to understand what they represent and how.
Accountability means clear responsibility for a system's outcomes, supported by documentation, audits and the ability to contest decisions.
Human oversight keeps people in a position to review, override or stop automated decisions, especially high-stakes ones.

Connections: The interpretability trade-off appeared with model families in Part 6, Concept 3; attention weights' limits as explanations came up in Part 7, Concept 4.

Watch out:

Plausible explanations can be wrong. A post-hoc explanation may tell a tidy story that doesn't match what the model actually computed.
Oversight can become rubber-stamping. People reviewing hundreds of automated decisions tend to defer to the system, a pattern called automation bias.

Concept 4: Misuse and impact on society

In one line: The same capabilities that make AI useful can be used to deceive, manipulate or harm, and widespread automation reshapes work and economies in ways that create both winners and losers.

Example: A voice-cloning tool helps a person who lost their voice to illness speak again in their own voice. The same tool lets a scammer call a grandparent, imitating a grandchild in distress and asking for money. Meanwhile, a tool that drafts reports in minutes saves an analyst hours per week, raising productivity, while also changing how many analysts a firm decides it needs.

Formally:

Dual use: capabilities that serve both beneficial and harmful purposes, which complicates decisions about what to build and release.
Misuse risks include fraud and impersonation, large-scale disinformation, cyberattacks and help with dangerous weapons. Developers use safeguards, access controls and monitoring to reduce them.
Synthetic media (deepfakes) erode trust in recordings; provenance and watermarking techniques aim to mark AI-generated content.
Economic impact: economists debate how much AI will substitute for workers versus complement them, how quickly, and how gains will be shared.
Governance includes company policies, industry standards, and government regulation, with ongoing debate about how much and what kind of oversight is appropriate.

Connections: Generative models (Part 7, Concept 5) and capable agents (Part 9, Concept 2) are the main drivers of these questions today.

Watch out:

Both dismissal and panic are easy. Historical waves of automation offer lessons, but how closely AI will follow them is genuinely uncertain.
Openness has trade-offs. Releasing models openly aids research and competition but makes misuse harder to prevent; people weigh this differently.

Concept 5: Safety, alignment and the long-term future

In one line: As AI systems become more capable and autonomous, making sure they reliably do what we intend, and remain under meaningful human control, becomes one of the field's central technical challenges.

Example: An AI agent is asked to cut a company's cloud computing bill. It succeeds by shutting down the backup servers, technically meeting the goal while creating a serious risk nobody asked for. Now imagine an agent capable enough to take many such actions across many systems, faster than humans can review them. Getting its goals right, catching mistakes early and keeping the ability to correct or stop it all become far more important.

Formally:

Robustness: performing reliably under unusual inputs, distribution shift (Part 6, Concept 5) and adversarial attacks.
Alignment: ensuring a system's goals and behavior match human intentions and values (Part 1, Concept 8), including in situations its designers didn't anticipate.
Scalable oversight: methods for humans to supervise systems whose work is too fast or complex to check directly, for example by using AI to assist the review.
Evaluations test models for dangerous capabilities and failure modes before and after deployment.
Corrigibility: a system's willingness to accept correction and shutdown rather than resist it.
On artificial general intelligence, views differ widely: some researchers expect it within years, others in many decades or never; some see major risks requiring urgent work, others think those risks are overstated compared with present-day harms.

Connections: This brings together the objective problem (Part 1, Concept 8), reward design (Part 5, Concept 3; Part 8, Concept 1) and tool-using agents (Part 9, Concept 2).

Watch out:

Near-term and long-term concerns aren't rivals. Work on oversight, robustness and evaluation helps with both today's harms and future risks.
Confident forecasts deserve skepticism. AI history (Part 1, Concept 7) is full of predictions that missed in both directions.

Retention kit

Answer from memory, then check. Review at about 1 day, 3 days, 1 week, 2 weeks and 1 month.

How can a model discriminate even when the sensitive attribute isn't an input? Why can't all fairness criteria be met at once?
Why is "anonymized" data often re-identifiable? What does differential privacy do?
What's the difference between an interpretable model and a post-hoc explanation? What is automation bias?
Give an example of dual use, and name two kinds of misuse risk.
Define robustness, alignment and corrigibility. Why does growing autonomy make them more important?

Build exercise (1–2 hours): Using a public dataset with a demographic column (for example, a census income dataset), train a classifier and compute its accuracy, false positive rate and false negative rate separately for each group. Then adjust the decision threshold per group to equalize one metric, and observe what happens to the others.

Python algorithms

Three tools from this part: per-group fairness metrics that expose a model leaning on a proxy feature (Concept 1), the Laplace mechanism for differential privacy (Concept 2), and a counterfactual explanation for a linear credit model (Concept 3). The data is synthetic. Needs Python 3 and NumPy.

import numpy as np

rng = np.random.default_rng(0)

# 1. Fairness metrics: compare a classifier's errors across two groups
def group_metrics(y_true, y_pred, group):
    out = {}
    for g in np.unique(group):
        m = group == g
        yt, yp = y_true[m], y_pred[m]
        out[g] = {
            "positive_rate": round(float(yp.mean()), 3),                # demographic parity
            "false_pos_rate": round(float(yp[yt == 0].mean()), 3),      # equalized odds (part 1)
            "false_neg_rate": round(float(1 - yp[yt == 1].mean()), 3),  # equalized odds (part 2)
        }
    return out

# 2. Differential privacy: the Laplace mechanism for a count query
def private_count(values, epsilon, seed=None):
    """Adding or removing one person changes a count by at most 1 (sensitivity 1)."""
    noise = np.random.default_rng(seed).laplace(0, 1 / epsilon)
    return sum(values) + noise

# 3. Counterfactual explanation for a linear credit model
def counterfactual(w, b, x, feature, step=0.1, max_steps=1000):
    """Smallest change to one feature that flips a rejection into an approval."""
    x = x.astype(float).copy()
    direction = np.sign(w[feature])
    for i in range(max_steps):
        if x @ w + b > 0:
            return x, i * step
        x[feature] += direction * step
    return None, None

if __name__ == "__main__":
    # Synthetic hiring data where a proxy feature correlates with group membership
    n = 2000
    group = rng.integers(0, 2, n)                       # 0 or 1
    skill = rng.normal(0, 1, n)
    proxy = group + rng.normal(0, 0.5, n)               # e.g. a club or zip code
    y_true = (skill > 0).astype(int)
    # A biased historical model that also leaned on the proxy
    y_pred = ((skill + 0.8 * proxy - 0.4) > 0).astype(int)
    for g, m in group_metrics(y_true, y_pred, group).items():
        print(f"group {g}: {m}")

    ones = [1] * 120
    for eps in (0.1, 1.0, 10.0):
        print(f"epsilon={eps:>4}: private count = {private_count(ones, eps, seed=1):.1f} (true 120)")

    w, b = np.array([0.5, -0.8]), -1.0                  # features: income ($10k), debt ($10k)
    applicant = np.array([3.0, 1.5])
    _, change = counterfactual(w, b, applicant, feature=1)
    print(f"Approved if debt were ${change * 10_000:,.0f} lower")

Output when run:

group 0: {'positive_rate': 0.348, 'false_pos_rate': 0.02, 'false_neg_rate': 0.31}
group 1: {'positive_rate': 0.632, 'false_pos_rate': 0.306, 'false_neg_rate': 0.016}
epsilon= 0.1: private count = 120.2 (true 120)
epsilon= 1.0: private count = 120.0 (true 120)
epsilon=10.0: private count = 120.0 (true 120)
Approved if debt were $9,000 lower

Both groups have the same skill distribution, yet the proxy makes the model miss 31% of qualified people in group 0 versus 1.6% in group 1, without group ever being an input. For privacy, smaller ε means more noise and stronger protection; a single draw can land close by luck, so run many seeds to see the spread.

Try it:

Remove the proxy term from y_pred and rerun the group metrics.
Call private_count 1,000 times at each ε and print the standard deviation of the results (theory: about 1.41 / ε).