About Lecture Notes
Return to
Astronomy Homepage

Scientific Method Lab


Science is not memorizing a whole bunch of facts.

My statement:

Science: Science is a methodical procedure for the exploring and understanding of nature.

According to the American Physical Society:

Science is the systematic enterprise of gathering knowledge about the world and organizing and condensing that knowledge into testable theories.

Further, good science depends on the willingness of scientists to:

Most of this quarter you'll be learning the results of the scientific study of astronomy.
But you won't always know how those results were obtained.
And you'll rarely do the science yourself.
Today we will do some science.

The ancient Greeks founded many scientific disciplines.
But the Greeks generally believed every question could be answered just using logic, clever arguments, and maybe some divine revelation.
Modern science still requires logic and cleverness but adds physical measurements and testing.
The idea of doing experiments to test hypothesis was developed by Galileo, Bacon, and others in the 1600s.
This new "scientific method" marked the end of the dark ages.
The power of the scientific method is clearly demonstrated by the incredible scientific advances of the last 500 years.


Comparison of Science and Religion

Goal of field:
SCIENCE: Determine the laws of nature
RELIGION: Spiritual enlightenment and improvement

How truths are determined:
SCIENCE: Scientific Method
RELIGION: Revelation and scripture
[Science doesn't actually deal with "truths", more on this shortly.]

How are followers convinced of truth:
SCIENCE: Repeatable experiments, logic
RELIGION: Faith

Status of truths:
SCIENCE: Provisional, can be disproven
RELIGION: Absolute, no proofs needed or desired

Note: Science and religion deal with different aspects of human existence, they do not have to conflict with each other.

Conflict arises when either subject infringes on the other's domain.
Religion should deal with moral and spiritual issues, it should not make claims on physical laws or facts.
Science should deal with physical laws, not claim moral or ethical knowledge.


Models

Model: An explanation of how or why something happens.

Hypothesis: An untested model.

Theory: A widely accepted model.

(That's different from the everyday usage of the word.)

No theory is ever accepted as absolute truth in science.
All theories are subjected to further testing.
We'll see examples in astronomy of theories widely believed and revered for hundreds of years before being shown incorrect.

Models and theories, not "truth", is what science produces.
Models are what I'll be teaching you, and I won't always be teaching the most accurate or most correct model!
The best and most accurate models can be quite complex.
To learn these better models, take more science courses!
Models are produced and judged through the scientific method.
Scientific models are judged in the most harsh and stringent way - they must predict and explain cold hard facts.


Scientific Method

There are many variations to the scientific method.
The following elements are common to most variations.

Problem:

The phenomenon to be explained or the questions to be answered.

Example: Why does the Sun shine?
How does the dog get out of the back yard?
How do we cure a patient with that type of cancer?

Hypothesis:

A model that explains the phenomenon, or answers the question.
First test, model should agree with already known facts and experiments done in the past. So scientists have to take lots of classes in part to learn this.

Example: The Sun is covered by burning gasoline.
Crawls under the fence through that small hole.
This drug will cure that cancer.

Predictions & Experiments:

Scientists should determine methods to test their hypothesis.
If a model cannot be disproved, it is not science.

Experiments are designed then performed
Models that fail the tests are modified or rejected.

Examples: Fires require oxygen and never burn as long or as hot as the Sun has been shining.
If block hole, dog won't escape.
We can give the drug to people with that type of cancer and see if they are cured. [Too simplistic, we'll learn how to design a realistic test later today.]

Theory: A hypothesis passing all its tests may eventually graduate to "theory" status.

All theories are subject to further testing.

 

Next we'll apply the scientific method to a simple question, and learn how to design good experimental tests.

 

Problem: Does talking to plants help them grow?

Hypothesis: A talked-to plant will grow more and better.

A good hypothesis would also explain why.

Experiment: Plant seed, talk to growing plant, see how it does.

 

What's wrong with this experiment?

How do we judge how well the plant is doing?
Compare to "normal" growth? What's normal?
If the plant does well, how do we know it was because of the talking?
Maybe your house is just a good environment for plants.

 

How do we judge how well the plant is doing?

Get a second plant, called the "control".
We'll talk to one plant but not to the control.
We can now compare the experimental (talked-to) plant to the control, we can even measure heights or weights to get a quantitative comparison.

 

Suppose the experimental plant does better, how can we be sure that it was because of the talking?

This is a very important question.
And there's much that must be done to make sure.
You must make sure that the plants are treated exactly the same in every way except for the talking.

 

Make sure both plants get equal sunlight (otherwise you don't know if the different growth was due to talking or sunlight).
Make sure they get equal water, equal size pots, same dirt, etc.
Make sure they were equally good seeds (or do multiple experiments and controls and hope the average seeds were equal).

 

You should not be allowed to care for the plants!

If you spend time talking to the plant:

You should not make the comparison measurements.

Comparisons should be done at a predetermined time. (If the talked-to plant wasn't doing better you might decide to "give it more time." Not equal treatment.)

Person doing comparison shouldn't know which plant is which.

 

The issues we're talking about are extremely important, the same questions apply to drug testing and much more.
In testing a new drug:
Some sick patients are given the drug and others (the controls) are not.
Patients should be randomly selected whether to get the drug.
Doctors and nurses should not know who gets what.

[Knowing can bias the results. If the doctor knows the patient isn't getting the real drug, they would be more likely and quick to consider that trial a failure.]

Patients should not know who gets the new drug.
Control patients are given a "placebo", something that looks the same as the drug but contains no medicine.
Patients who think they're getting medicine do better.
The psychological impact (the hope of a cure) is enormous.
This is called the "placebo effect".

This is called a "double-blind" study.
Neither the doctors nor patients know who gets the drugs.
Only then can you be sure that if patients getting the real drug do better that it was because of the drug.
Double-blind studies are considered the "gold standard" of drug tests.

 

Now getting back to our plant experiment:

There are still more precautions that must be taken to prove our hypothesis.

 

* Random chance

The talked-to plants might do better just by chance.
[It is very common for scientific studies to result in statements like, "Such-and-such is true with 95% confidence.]

 

* Simultaneous Effects (unaccounted-for factors)

When talking, you exhale CO2 (carbon dioxide) and moisture.
Plants love that stuff, it helps them grow.
The talked-to plant might do better, not because of your talking, but because of your breathing onto it.

Solution?
Have plant talked to by a tape recorder?
Breathe equally on other plant without talking?

[That is, talking to plants probably does(!) help them grow better, but it's all these other simultaneous effects, not the talking.]

 

* Experimenter Bias

Maybe you do the experiment 10 times.
Five times the control did better and you threw out that data because it was obvious that it was faulty.
The other trials all had the experimental plant doing better so you conclude that talking really does help plants grow.
Or you exclude a talked-to plant that died early on, obviously a bad seed. (That's why people trust double-blind studies, you can't be biased if you don't know which plant is being talked to.)

 

This was supposed to be a simple experiment.
But good experiments, where you control all the variables can be quite challenging.
And plants are a lot easier to deal with than sick people!
Hopefully you are getting a feeling for how science is done.


It is hard to do good science!
Because it is very easy to convince yourself of the answer you expect or desire.
You should distrust scientific claims from biased sources.

 

Astronomy experiments are usually different.
Astronomers can only observe outer space, rarely control or manipulate it.
Not all things can be tested with double-blind experiments.

Some astronomy experiments are like what we've described.
We'll do such an experiment now.


Astrology Testing

Astrologers claim to be able to predict what will occur to you every day based simply on where the Sun was when you were born (your "Sun-sign" or just "sign").
In fact, most newspapers print a daily column of astrological predictions.

Handouts.

On the handout, write down your
Name, birthday, astrological sign
Before looking closely at the horoscopes listed, think about what happened to you yesterday.
Relationships, money, acquaintances, journeys, arguments, etc.
Now take your time, read all the horoscopes and pick the one that best matches the day you had yesterday.

Questions?

 

You can only pick one horoscope. You must pick one and only one.
Everyone check your neighbors to make sure.
You are not being graded on whether you get it right.
Reveal correct signs for each horoscope.

If astrology is correct, we might expect that 20 or 30 people would pick their correct horoscope. But if this astrologer really doesn't know what he is doing, then it would be just chance whether a person guessed the correct horoscope.

By random chance, we would expect about 5 (maybe 3 or 9 or 6) to guess correctly.
20 or more correct - astrology works, 0 to 10 correct - astrology fails.
Raise your hand if you guessed correctly.

We haven't fully disproven astrology, but astrology has never passed a scientific test of its effectiveness.

 

Now for a second test of astrology.

Personality traits are supposed to be, according to astrology, at least partly determined by one's Sun-sign.
It requires certain personality traits to become U.S. president (leadership, charisma, out-going).
So we might expect most presidents to have the same sign (or one of a few).

By random chance, each sign should have 3, 4, or 5 presidents (but any number from 0 to 8 is likely).
If there is a correlation, we'd expect many signs to have 0 presidents and a few to have 5 to 10 or more presidents.

Count Presidents with each sign.

Answer the questions.

Turn in the handouts! (Else you won't get attendance credit.)



Return to
Astronomy Homepage