Friday, August 04, 2017

Seth Shostak and the Odds of SETI Success

"If this project is going to work at all it's going to work before you all become middle-aged." (Seth Shostak)

Seth Shostak thinks that, assuming SETI has the basically right idea, we will likely detect a signal from an alien civilization by 2035. It's important to keep in mind that this probability implies another: it is likely that a signal from an alien civilization will reach us during the next two decades. Indeed, I would argue that Shostak must believe that it is likely that the signal is already hitting us and we just haven't detected it yet. I think this assumption underpins all work in the SETI area: at any given moment it is very likely that a signal, though it may be difficult to detect, is striking the surface of the Earth.

According to the standard model, we're searching about 200 billion stars for somewhere between 10,000 and 1,000,000 "advanced technical civilizations", which Carl Sagan "operationally" defined as "societies capable of radio astronomy". Depending on how many civilizations there in fact are, Shostak argues, and given Moore's law of technological development, we should find the first of them by 2035 at the latest. That is, he is not thinking about the probability that a signal is hitting us; he is thinking about the probability of detecting one of the signals that, he assumes, is hitting us. If there are many, it won't take long. If there are few, it will take longer. But at some point we will have searched the entire probability space of the "cosmic haystack".

But consider our own detectability. Our ambient signal "leakage" from TV and radio (which would be very hard to distinguish from noise in any case) only reaches about 80 light years into space. There are only about 500 sun-like stars in that space and many more stars beyond it. The galaxy is 100,000 light years across. Our signal occupies hardly any of it. It gets even worse when we consider the few attempts at an intentional signal we've sent. In 1974 we sent the "Arecibo Message" for about three minutes. About forty light years away from Earth now, it occupies a tiny sliver of space only three-light minutes long. If it ever does hit a civilized planet, they have to be listening at the exact time that it does. If they blink for three minutes, they'll miss it.

So let's think about the space between us and any one of those "advanced technical civilizations". It is between 4 and 87,000 light years long, with more and more of the stars we're looking at being further and further away. Only .3% of the stars in the galaxy are within 5000 light years of us. And the closer the civilization is to us, the more likely it is that the signal hit us at the time of Socrates and is now a thousand light years away from us and receding! Moreover, we don't know when they might start signalling—a million years ago? A million years from now?

In other words, a million advanced technical civilizations cannot mean a million signals in our sky just waiting to be discovered like any other astronomical object. We can detect the light of stars and galaxies because it has been shining on us for billions of years and will continue to shine for billions more. But any imaginable signal from a technical civilization will have a finite duration. It is, after all, an artifact, a product of the history of the alien civilization. In my view, SETI forgets that alien intelligence, like ours, will be historically situated. When we have not found anything before Shostak's audience reaches middle-age it will prove, as he says, that it's not going to work at all. But we don't have to wait that long to realize that the project doesn't make any sense even on paper.

Wednesday, August 02, 2017

The SETI Coat of Arms

Somewhere along the line of my career I started to think, "Okay, maybe I won't find the signal (I can do the best job I can [but] I really can't control that). But if I can leave this field financially stronger than when we started—if we can find a way, by the time that I'm finished cheer leading, to have a stable financial funding for this kind of exploration—which may, indeed, be multigenerational—then I will have done something pretty damned good. And I can feel very good about that." (Jill Tarter)

This remark during last month's SETI Talk about Sarah Scoles' biography of Jill Tarter reminded me of a short story by Franz Kafka called "The City Coat of Arms". It is, of course, well worth reading in its own right and it is only about 500 words long so you might want to go ahead and find it and read it before reading the rest of this post. I don't want to spoil your enjoyment.

The story is about the building of the Tower of Babel. In fact, it's not so much a story as a page from a fictional history book. After noting that the building arrangements were "perhaps too perfect", our historian establishes the central premise:

The essential thing in the whole business is the idea of building a tower that will reach to heaven. In comparison with that idea everything else is secondary. The idea, once seized in its magnitude, can never vanish again; so long as there are men on the earth there will be also the irresistible desire to complete the building.

Anyone familiar with the rhetoric of SETI will see the connection. As Carl Sagan put it in his famous essay, "The Quest for Extraterrestrial Intelligence" (which, incidentally, uses a Kafka quote about the silence of the Sirens as its epigraph),

The search for extraterrestrial intelligence is the search for a generally acceptable cosmic context for the human species. In the deepest sense, the search for extraterrestrial intelligence is a search for ourselves. [...] It is difficult to think of another enterprise ... that holds as much promise for the future of humanity.

But Kafka understood, as Tarter also has come to understand, that once the, let's say, infinite promise of the project has taken hold of the imagination it does not need to succeed in any foreseeable future in order to retain its relevance, or, as Tarter rightly notes, its funding. Indeed, its significance for humanity will come from the silence, not the song, of sirens. Like Tarter, Kafka also seizes on the multi-generational aspect:

[O]ne need have no anxiety about the future; on the contrary, human knowledge is increasing, the art of building has made progress and will make further progress, a piece of work which takes us a year may perhaps be done in half the time in another hundred years, and better done, too, more enduringly. So why exert oneself to the extreme limit of one's present powers? There would be some sense in doing that only if it were likely that the tower could be completed in one generation. But that is beyond all hope.

Here, some SETI enthusiasts will say that the analogy breaks down. It is not they will insist "beyond all hope" that SETI will succeed in a generation. Yuri Milner has committed $100 million dollars to a ten-year push to find whatever there is to find—the Breakthrough Listen project. But I would encourage them to read a recent paper by Claudio Grimaldi. He shows quite convincingly that the probability of detecting a signal, even if the galaxy is teeming with intelligent life (Sagan estimated a million civilizations capable of radio astronomy) is exceedingly small. This is because the signals themselves, by definition transmitted on a "historical" timeframe, disappear in the astronomical volume of the galaxy. They simply can't fill in any reasonable volume of galactic space to make it very likely that the earth's orbit will pass through the decidedly finite volume occupied by the signal.

It is only if the alien civilization had in fact undertaken to build a lighthouse to reach the heavens, to signal continuously in all directions for millions and millions of years, that their signal would have any chance of reaching us at our particular moment in history, i.e., the ten-year funding framework of the Breakthrough Listen project. So Tarter is right, something more permanent needs to be established. But here, too, Kafka must be heard:

It is far more likely that the next generation with their perfected knowledge will find the work of their predecessors bad, and tear down what has been built so as to begin anew.

Again, there are echoes of the SETI rhetoric. We need to "speed up the search," as Shostak puts it. We need more powerful computers "on the back end" to analyze the data. We might not even know, yet, what to look for. The aliens might be messaging to us using, not radio waves, but laser beams or neutrino streams. We just don't know! On the one hand, this cornucopia of possibilities gives us hope. On the other hand, the hope is distributed across a probability space that requires millions of years (of listening and of sending) to achieve reasonable "coverage" (to use Grimaldi's term). "Such thoughts paralyzed people's powers," Kafka says of the builders of Babel,

and so they troubled less about the tower than the construction of a city for the workmen. Every nationality wanted the finest quarters for itself, and this gave rise to disputes, which developed into bloody conflicts. These conflicts never came to an end; to the leaders they were a new proof that, in the absence of the necessary unity, the building of the tower must be done very slowly, or indeed preferably postponed until universal peace was declared.

This is where it gets real for me. It is important to remember that SETI is not just a scientific field but a political situation. It must lobby for resources, as Tarter rightly points out, and it must manage its own internal conflicts and intrigues. Sarah Scoles, it should be remembered, has written not just about Jill Tarter but also about Geoff Marcy. She is not just interested in the former's heroic quest but that latter's infamous transgressions. She has, as it were, one auspicious and one drooping eye on astronomy. She is interested in the struggle for both truth and justice. She covers both the scientific discoveries and the political conflict.

"The time," says Kafka,

was spent not only in conflict; the town was embellished in the intervals, and this unfortunately enough evoked fresh envy and fresh conflict. In this fashion the age of the first generation went past, but none of the succeeding ones showed any difference; except that technical skill increased and with it occasion for conflict.

But, throughout it all, surely there is Sagan's "enterprise"—that great hope for humanity—animating these scientific and political projects? Well, one sometimes wonders. As Kafka puts it:

[T]he second or third generation had already recognized the senselessness of building a heaven-reaching tower; but by that time everybody was too deeply involved to leave the city.

It's this deep involvement that I discern in Tarter's remark, which she specifically frames in terms of her "legacy". It is no longer important to discover a signal. Indeed, Grimaldi may be right and the entire enterprise may be senseless. (I have my own back of the envelope calculation that suggests something similar.) What is important is to embellish "the city", to leave the field "financially stronger", as Tarter puts it, than we found it.

Remember that it may take generations. Tarter is still part of the first generation—those who came up with the idea of finding an intelligent signal and might yet die trying. There will now be a second and third generation, whose efforts may also fail, but will involve a great deal effort to justify increasing investments in time and technology. The odds for them will not improve markedly. In order to beat Grimaldi's house you need to listen for millions of years for signal that has been transmitted for just as long. Kafka anticipates the cultural impact of despair on such an astronomical scale:

All the legends and songs that came to birth in that city are filled with longing for a prophesied day when the city would be destroyed by five successive blows from a gigantic fist. It is for that reason too that the city has a closed fist on its coat of arms.

Or, in our case, all the movies are filled with gigantic asteroids that threaten humanity with extinction. Less dramatically, the SETI Institute's logo depicts, not so much a signal, as a question. Perhaps it should consider a fist with the tail of a comet?

Monday, July 31, 2017

Looking For Bowhead Whales One Glass of Ocean at a Time

In a recent SETI Talk celebrating the publication of Sarah Scoles' biography of Jill Tarter, Seth Shostak reminded me of a puzzling analogy that Tarter likes to use to explain why we haven't found a signal from an alien civilization yet.

Jill likes to say that [if] you go to the ocean and take out a glass of water and you don't find any bowhead whales or something [you wouldn't] conclude that there aren't any whales in ocean. She's emphasizing the fact that the sample size has been very small. (29:51, lightly edited.)

In this analogy, however, it's not so much the sample size as the sampling rate or resolution of the search that is the problem. You can't catch a bowhead whale with a highball glass; so you're looking for something in a way that precludes you from finding it. I think other SETI researchers sometimes jokingly use the parable of the drunk who's looking for his keys under the street light. When asked where he lost them, he points down the street a ways. "Why are you looking here then?" we ask. "Because the light is better," he replies. This is not a joke SETI researchers should be telling. It's literally on them.

Since Tarter is a woman, the same SETI talk begins with an obligatory discussion of gender discrimination in science. I hope women in science will soon band together against this theme—the obligation in particular. Let them talk about their struggle to discover the truth, not their struggle as women. But I digress. Tarter addresses this topic by talking about what happened at Starmus this year, when she was, shockingly, exposed to gendered humor. I ended my post about that incident with a jab at Tarter's demonstrated inability to sort signal from noise. In my view, gender activists are not looking for bowhead whales one glass of water at a time—though I think SETI is looking for aliens that way, I'm afraid. Rather, their null hypothesis seems to be finding pure H20 in a glass of raw sea water. In their surveys, of course, they're constantly finding it full of salt and life. Shocking!

Thursday, July 27, 2017

Is Astronomy a Hostile Workplace for Women and Minorities?

The authors of a recent paper in the Journal of Geophysical Research think so. In a survey they conducted in early 2015, astronomers reported how often they heard negative language, experienced verbal and physical harassment, felt unsafe, and skipped work activities due to concerns about their safety.

88% of respondents reported hearing negative language from peers at their current career position, 51.9% reported hearing negative language from supervisors, and 88% reported hearing negative language from others. Thirty-nine percent of respondents report experiencing verbal harassment at their current position, and 9% report experiencing physical harassment. Twenty-seven percent of respondents report that they have felt unsafe at their current position, and 11% report that at their current position they have skipped attending at least one professional event such as a class, meeting, conference, or fieldwork opportunity because they felt unsafe attending.

They go on to say that their results "suggest there is not only a hostile climate in the astronomical community but that the community is experienced differently depending on one’s gender and race." The paper has received broad coverage in the press,* where "widespread harassment" in astronomy is now reported as an established fact. "The sciences are overwhelmingly hostile to women," wrote Rae Paolotta at Gizmodo, "and in astronomy, it’s doubly bad for women of color." Kate Clancy, the lead author of the paper, called it "great piece" on Twitter.**

The astronomy community appears not to have any objections to this characterization. The American Astronomical Society and the American Geophysical Union issued a joint statement, acknowledging the findings and promising "positive change". When I asked the executive director of AAS, Kevin Marvel, to comment specifically about the paper's claim that there is "a hostile climate in the astronomical community" he offered the following response:

We are glad the report has come out and it is now important for the community think carefully about the information it presents and the resultant recommendations. Although we have made good progress in some areas regarding harassment, there is more to do, and reports like this play an important role in moving the ball forward as we say in the US. We will use this, and other sources of information and recommendations to constantly work toward ensuring a professional environment free of harassment. Let us hope we achieve it sooner rather than later.

As far as I know, no one other than me has publicly challenged the assertion that astronomy is a hostile workplace for women and minorities. If the representatives of the astronomy community wanted to defend its members and its culture against this charge it could easily do so by looking more closely at the results. The AAS's Committee for the Status of Women in Astronomy sponsored the survey, and one the authors is a member, so I assume they have access to the raw data. As I never tire of pointing out, this would allow them to disaggregate the results according to the never/rarely/sometimes/often scale of the questionnaire. Since the authors of the study completely ignore me, I have been estimating what this would reveal based on the slides of the preliminary results that were presented at DPS in 2015 and AAS in 2016.

Here is why I don't believe the hype. Although 39% reported verbal harassment, less than 13% appears to have reported it occurring more than "rarely", and less than 2% seem to have reported it happening often. That's about seven people in a sample with a strong self-selection bias towards people who have something to report, including witnesses and allies, and an (intentional) oversampling of women. In the most extreme self-selection scenario we can imagine, everyone in the astronomy community who feels they're often harassed would have reported. In that case, .07% of astronomers (pop. 10,000) experience verbal harassment often. I estimate the upper bound on the amount who experience verbal harassment often at 60 people (2% of female astronomers); in that case, about .6% of astronomers (men and women) experience verbal harassment often.

A similar approach applies to all the other results. First we disagreggate the total percentage into never/rarely/sometimes/often and discover that a majority of respondents in the sample report the measured experiences of "hostility" never or rarely. Next, to get a realistic sense of prevalence in the whole population of astronomers, we adjust (always downwards) for the self-selection of victims, witnesses and allies, and the oversampling of women.

I believe astronomers are being let down on multiple fronts. Journalists are clearly not covering this story in an impartial, or even competent, manner. They are not applying even a modicum of skepticism to some obviously sensational claims being published with obvious political ambitions. They don't seem to have even a basic understanding of sampling bias or the now very well-known problems associated with significance testing. (We'll get to that in another post.) Social science is also letting astronomers down by making overblown claims based on underpowered studies and promoting their spread through the media. The journal editors and reviewers here also don't seem to have thought about the reputations of astronomers, either as a field or as individuals, when accepting this study for publication (and promoting it thereafter). Finally, I think the political bodies that are supposed to tend to the interests of astronomers have let their membership down. Not only did the AAS fund this study, they have not offered any critical pushback on behalf of the community it smears as steeped in sexism and racism.

All this, of course, is just my opinion as an outsider looking on a field that I used look at with awe and envy. I hope astronomers will find a way to get their house in order. To riff on Kevin Marvel's statement: let us hope they achieve it sooner rather than later. I am here to help in any way I can.

____________
*Here are some representative headlines: "Women of color face staggering harassment in space science" (Sarah Kaplan, WaPo). "Women of Color in Astronomy Face Greater Degree of Discrimination, Harassment" (Calla Cofield, Space.com). "Survey data point to widespread problems" (Colleen Flaherty, IHE). "There’s a lot of bias in astronomy" (Angela Chen, The Verge). "Female astronomers of colour face daunting discrimination" (Rachael Lallensack, Nature). "A new survey of astronomers and planetary scientists reveals a workplace harassment problem in the space sciences" (Francie Diep, Pacific Standard). "Unprecedented study reveals widespread bias in space science, and it's particularly terrible for women of color" (Miriam Kramer, Mashable). "Astronomer survey reveals gender and racial harassment" (Michael Banks, Physics World). "Widespread harassment reported in astronomer survey" (Toni Feder, Physics Today).
**Update: I forgot to mention that Clancy endorsed Paelotta's article even though it got the study's methodology completely wrong. "Clancy and her team surveyed 474 astronomers and planetary scientists between 2011 and 2014," says Paolotta, though the study was conducted from January to March of 2015. "All subjects identified as women or non-binary," she also says, though the sample was almost one third male.

Tuesday, July 25, 2017

Exclusion

Here's question 7 of the CSWA Workplace Climate Survey:

In your current position, how often have you been VERBALLY harassed because of the following characteristics?

{Choices include: Often, Sometimes, Rarely, Never}
Race or Ethnicity
Physical Disability Status
Mental Disability Status
Sexual Orientation
Gender Identity(Cisgender or Transgender)
Gender (Female, Male, or Non-binary)
Religion or Lack Thereof

Question 9 has the same form, with "PHYSICALLY" replacing "VERBALLY". Now, here's hypothesis 1:

Female respondents will report more verbal and physical harassment than men.

This sentence appears in the first paragraph of the results section:

Thirty-nine percent of respondents report experiencing verbal harassment at their current position, and 9% report experiencing physical harassment.

And this one appears in their support for hypothesis 1:

Women were also significantly more likely than men to report that they experienced both verbal and physical harassment because of their gender.

I have underlined that phrase because it draws attention to the glaring absence in the questionnaire of a "characteristic" that is likely to have been focus of harassment directed at white male astronomers. Indeed, though the paper doesn't tell us this (we know it only from Christina Richey's preliminary presentations at DPS in 2015 and AAS in 2016), race and gender account for the great majority of characteristics that people felt they had been harassed for. In the case of verbal harassment, they account for 65% of the reports. In the case of physical harassment they account for more than 80%.

For obvious reasons white male astronomers are not likely to report being victims of harassment because of their race or gender. If they are also straight, cisgendered, protestant, able-bodied and do not suffer from mental illness, they would seem to have no way to report their experiences on the survey. And yet, surely, they might experience harassment. Most commonly, they will experience verbal (and at times physical) harassment by professional rivals, for whom they are competing for publication, promotion and research funding. This basis for harassment has been completely excluded from the Workplace Climate Survey. The questionnaire did not even provide a generic "other" characteristic in which to report harassment.

Now, if the hypotheses tested had confined themselves to race- and gender-based forms of harassment, this wouldn't be such a big problem. Except that the conclusion that women and people of color experience more gender- and race-based harassment than white men is a bit underwhelming. But Clancy et al. claim to have found support for hypothesis 1, and they are promoting the result widely as suggesting that women, and women of color, experience more harassment than men full stop. As it turns out, this conclusion emerges from a measurement instrument that excluded the most typical experiences of harassment among white men.

I'm going to take some time to think about the consequences of this. But my initial reaction is that it completely undermines the validity of the survey, given the hypotheses it claims to be testing. Also, it raises the interesting question of whether women and minorities experience harassment based on professional rivalry (they would also not be able to report it). And that, finally, raises a question that concerns me greatly: is it possible that women and minorities are getting the axis of their harassment wrong? Is it possible that they experience harassment that is really grounded in ordinary competition as grounded in racism and sexism? If so, surveys like this are distracting them from the fight they should be fighting; and this, ultimately, will be to their disadvantage.

Comments are welcome, as always.