Author Archives: kevin

About kevin

I write the posts

Proposed SSF East Side Station Exit Prioritizes Cars, Not Caltrain Passengers

Posted on

Previously I wrote about pedestrian/bike improvements for the east side of 101. There's now a proposal for the east side exit of 101, shown in this slide at a recent Caltrain board meeting.

It's not ideal to have pedestrians and cyclists on the north side of East Grand for two reasons: ped/bike crossings will hold up shuttles merging onto East Grand, and peds/bikes have to go out of the way up to the East Grand/Grand intersection and then cross two signals - one with a 30 second wait time and the other with a 42 second wait time. If one of the sidewalks in that photo is supposed to be a bike path, it's currently in poor shape and it's not clear there are plans to repair the surface.

It's pretty easy to imagine instead that cyclists will try to get across 2-3 lanes of traffic here, instead of waiting for two WALK signs at East Grand:

In an ideal world, bikes/peds could just cut straight across to the Gateway/East Grand intersection like this. This would cut at least a minute off of pedestrian travel times and extend the walkshed that much further. I believe SSF owns the right of way where the bottom red line is located.

Could SSF, Caltrans, and SamTrans staff advocate for a ped/bike crossing on the south side of East Grand, that would allow ped/bike to travel on the south side of the intersection, saving a circuitous route for bikes and pedestrians? Ultimately this will require a decision that it's more important to prioritize Caltrain riders, even if it means a slightly longer delay for car traffic.

The city of South San Francisco is the ultimate decision maker here, but they would appreciate your input on good design outside the station area. Contact the City Council: council@ssf.net and the Bike/Pedestrian Advisory Committee: BPAC@ssf.net.

Liked what you read? I am available for hire.

Research on railway suicides and steps Caltrain could take

Posted on

It's Rail Safety Month and last month at the Caltrain CAC we got a presentation on suicides on the Caltrain line. I've added some key papers that study railway suicide to this Github repository. At a high level, they show:

  • There are several factors that predict suicide incidence.

  • There are several factors that have been demonstrated to reduce suicide incidence.

Together these suggest that Caltrain may be able to take steps to reduce the incidence of suicide. Caltrain has no dedicated source of funding, and staff time is extremely valuable, so it's understandable that this hasn't been a focus area, and it's also extremely likely that it won't be possible to make improvements in these areas for some time.

What factors predict suicide?

  • Psychopathy/psychiatric hospitalization. An above average number of victims have a documented psychiatric history.

    One paper suggests that track areas near psychiatric facilities are higher risk areas for attempted suicides. We could map Caltrain's passenger strikes against psych facilities on the Peninsula corridor, and invest in fencing in those areas.

  • Easy access to train tracks: In many cases railway suicide attempts are impulsive and not very well thought out. "24% of the people who made near-lethal suicide attempts took less than 5 min between the decision to kill themselves and the actual attempt, and 70% less than 1 h," according to Miller and Hemenway (2008).

  • Frequency of train service: The more trains that run the more suicides there are. This is a risk factor for electrification, which plans to increase rail frequency.

  • Knowing someone who has committed suicide: there is a contagion effect where one suicide may lead to others; either sensational coverage of a suicide or a community member committing suicide may prompt copy cats.

What are the costs of suicide?

  • Loss of life: Branches of government like the US DOT assign a statistical value between $5 million and $10 million to a single life when evaluating safety improvements. This suggests an investment of $3-4 million in railway safety is a net positive if it saves a single life.

  • Loss of railway service: Train strikes frequently result in death, and trains cannot begin moving again until a coroner can come on site to evaluate the cause of death. This can result in delays of several hours. This results in lost revenue — commuters may choose not to ride the train if the arrival time is unpredictable — and lost productivity from missed meetings, work time and so forth.

  • Employee trauma: The psychological effects of striking people on the railway have not been as well studied, but some drivers experience severe reactions to striking people. This increases the cost of hiring and trauma management.

What measures can prevent suicide?

Highly effective:

  • Platform Screen Doors: Subways in Singapore and Hong Kong saw a significant reduction in suicides - 59% - after installing screen doors at some stations. Suicides did not increase at other stations without screen doors, either, which fits with a characterization that suicide is an impulsive behavior, and deterring people with a suicidal impulse is enough to lower total rates of suicide.

    Tasha Bartholomew suggested that on the Caltrain line about half of suicides occur in station areas and half do not. The requirement to have freight traffic on the corridor and potential FRA regulations around such may also make it impossible to add platform screen doors. Still, we could evaluate the possibility of platform doors at new stations or identified hotspots.

  • Media Training: Between 1984 and 1987 there was dramatic coverage of subway suicides in Austria and a corresponding spike in suicides. In 1987 a group developed media guidelines for suicide coverage and the suicide rate promptly dropped by 80%. Suicide rates by other means also declined suggesting no substitution.

    Media coverage is more likely to lead to suicides when:

    • Lurid details about the method are provided

    • The more it's reported as being inconceivable ("he had everything he could have wanted")

    • If it's reported as having a romantic motive

    • If it's on the front page, the word suicide is in the headline, a photo is provided

    These steps can reduce suicide incidence:

    • Telling the audience where to find help, providing background information about suicidal behavior, and what to do with people with suicidal thoughts.

    The Caltrain social media team should be praised for speaking about passenger strikes and trespassers in generalities - never providing details and only providing photos/video in incidents that did not lead to injury.

    There are only a few reporters and outlets that cover Caltrain regularly so working with them may pay dividends if we are not already doing it.

Somewhat Effective:

  • Reducing access to the railway more broadly: This can come in many forms besides platform doors - grade separations, more fencing, ticketed station areas, identifying hot spots like mental hospitals and increasing fencing there.

    Several places on the Peninsula are considering grade separation projects — Palo Alto, Burlingame, others — and reducing pedestrian access to tracks should be a key consideration above and beyond grade separating cars and trains.

  • "Suicide pits" (deep channels between rails): Some stations in the London Underground have a pit three feet deep between the rails at stations. This allows people who have second thoughts to duck below the train, and reduce the potential for the train to drag people along with it, and thus cause further injury. Strikes at Underground stations with pits are less likely to lead to fatalities than stations without them.

    This may be worthwhile to install at new stations such as the Hillsdale station (if it is not already being considered).

Needs More Study:

  • Train design: Fatal pedestrian/car accidents have been on the rise due to Americans buying larger cars with a higher front profile. Similarly, the design of the train engine may make strikes more or less deadly depending on the design. A lower point of impact and a skirt which makes it impossible for a body to go underneath the train may reduce the likelihood of a fatal injury.

    An airbag that was deployed concurrently with the emergency brake could also reduce the impact of a collision. There is not enough research into the effect of these changes, though they may be cheap to implement relative to grade separations or screen doors.

    Given the lack of research here, perhaps a rail safety study sponsor could fund a modified skirt design on half of the new engines that Caltrain is ordering. We could then measure the injury and fatality rate of train strikes with the modified skirt and the original design.

Ineffective:

  • Blue lights: A study in Japan got attention for saying that blue LED lights at station platforms led to a lower number of suicide attempts. A later study found that the effect of blue lights was not measurable.

Liked what you read? I am available for hire.

Bike and pedestrian improvements east of the South San Francisco Caltrain station

Posted on

For the past month or so I've been commuting to a biotech office east of 101 in South San Francisco. I've gotten a little depressed about how car-centric the infrastructure is and I wanted to share some small wins that could improve the bike and pedestrian experience.

Why bother? Well, the roads get really full at commute time and it's not easy to expand them. People biking and walking (5 sq ft and 2 sq ft) take up less space than they do in a car (150 sq ft), so you can fit more people on the roads. They're also faster - I routinely get to Caltrain faster on a bike than I would be able to in a car, because the bike lane is empty.

However, the experience is currently not great, which means it's scary for a lot of folks to bike. Here are some small things that South San Francisco could do to make it easier for pedestrians.

  • Put WALK signs in front of the limit line: This WALK sign at Forbes and East Grand is behind the limit line.

    If a bus or truck is in the first spot - which is common, Genentech runs shuttles quite frequently - you can't see the sign or figure out when the WALK sign is on. Moving the WALK sign in front of the limit line would increase visibility, and also...

  • Put pedestrian crossing buttons closer together. It's at least 20 feet between these WALK buttons at East Grand and Gateway.

    If you are trying to cross the street diagonally - which will become more common when the Caltrain station moves - you have to walk a long way to press both buttons. Better would be to put the buttons on the same pole. This would also move the WALK sign ahead of the limit line.

  • Prioritize pedestrian traffic. Often you have to wait a long time to cross the street after pressing the button. The crossing could stop the flow of cars soon after a pedestrian signals their intention to cross the street, or stop more often in the minutes before a Caltrain is scheduled to depart from the station. If we want to encourage pedestrian traffic we should not make walkers wait as long at intersections.

  • Narrow car lanes and make better bike lanes. If you want to encourage cycling you want to have networks, where people can bike places and feel safe the whole way. There are some bike lanes east of 101, but they don't form a network, or allow you to go where you are trying to go.

    The bike lane starts at the bottom of the bridge over the train tracks. The outside lane of East Grand at this point is 21 feet wide. This is enough room for a 2 foot gutter, 4 foot bike lane, 2 feet of protection, and a 13 foot car lane. Adding a bike lane in this stretch would make bicyclists feel safer.

    Between Gateway and Roebling St. the outside lane of East Grand is 16 feet and the inside lane is about 13 feet. Making each of those 11 feet would give enough room for the gutter and a 4 foot bike lane. 11 feet might feel cozy for a "main arterial" but again - cars aren't going that fast on this road anyway due to the high number of stoplights and other cars on that road.

  • Integrated plan for getting walkers/bikers from the new Caltrain undercrossing to Verily/Genentech/Pfizer etc: There are plans for a new station platform and a undercrossing which comes out near the Poletti Way offramp, but after that there are lots of unresolved questions. The SSF "Downtown Station Area Specific Plan" merely says "design will need to consider safe access" without specifying what the design will end up being.

    If I am biking from downtown South San Francisco to Verily, Pfizer or Genentech, should I take the undercrossing - is the undercrossing wide enough for both cyclists and pedestrians? - or is the Grand Ave bridge still the recommended way to go?

    What is the plan for getting pedestrians and cyclists across the Poletti Way offramp and to the biotech campuses? Some ideas:

    • Extend the tunnel under the Poletti Way offramp. Otherwise, add a speed table and a wide curb bulbout to make it easier and safer for pedestrians to cross Poletti Way. Here's an example of what this could look like:

      A scramble intersection could also work there.

    • Extend the bike lane on Gateway north of East Grand, a bike lane that's in the Bicycle Master Plan. Does the city own the land east and west of the roadway? This could be used to add bike lanes without making changes to the roadway.

    • Remove the eastbound parking lane on East Grand near the Caltrain undercrossing, then add a bike lane + pedestrian path next to the tree line, straight across East Grand to Gateway.

  • Remove setback requirements: Most of the offices and stores are set back from the street across acres of parking.

    These are unfriendly to pedestrians, since you have to walk further to get where you are going. Allowing stores and offices to be built right up against the property line will reduce walking times, allow pedestrians to stop to shop on their way to and from work, and encourage walking and cycling.

Kevin Burke is on the Caltrain Citizens Advisory Committee.

Liked what you read? I am available for hire.

AWS’s response to ALB internal validation failures

Posted on

Last week I wrote about how AWS ALB's do not validate TLS certificates from internal services. Colm MacCárthaigh, the lead engineer for Amazon ELB, writes:

I’m the main author of Amazon s2n, our Open Source implementation of TLS/SSL, and a contributor to the TLS/SSL standards. Hopefully I’m qualified to chime in!

You’re right that ALB does not validate the certificates on targets, but it’s important to understand the context that ALBs run in to see why this is still a pending item on our roadmap, rather than something we’ve shipped already as a “must have”.

The role that server certificates play in TLS is to authenticate the server, so that it can’t be impersonated or MITM. ALBs run exclusively on our Amazon VPC network, a Software Defined Network where we encapsulate and authenticate traffic at the packet level. We believe that this protection is far stronger than certificate authentication. Every single packet is being checked for correctness, by both the sender and the recipient, even in Amazon-designed hardware if you’re using an Enhanced Networking interface. We think it’s better than the ecosystem where any CA can issue a certificate at any time, with still limited audit controls (though certificate transparency is promising!).

The short of it is that traffic simply can’t be man-in-the-middled or spoofed on the VPC network, it’s one of our core security guarantees. Instances, containers, lambda functions, and Elastic Network Interfaces can only be given IPs via the secure and audit-able EC2 APIs. In our security threat model, all of this API and packet level security is what plugs in the role performed by server certificates.

This contrasts with the older EC2 classic network, a big shared network, which is why classic load balancers do support backend authentication.

We actually find that many customers actually load their targets and backends with “invalid” certificates that are self-signed or expired, because it’s so operationally hard to stay up-to-date and it’s hard to automate, even with projects like LetsEncrypt, when your instances are inherently unreachable on the internet.

All that said, we’ll be adding support for certificate validation, probably including pinning and private CAs! Used well with good operational controls it can be a measure of defense in depth, and it’s important for cases such as targets hosted on less secure private networks such as on-premesis data-centers.

Liked what you read? I am available for hire.

Amazon’s ALB’s do not validate TLS certificates from internal services

Posted on

If you are using an Amazon Application Load Balancer, and forwarding traffic to internal services using HTTPS, the ALB will not validate the certificate presented by the internal service before forwarding the traffic.

So we're clear here, let's say you are running a web server on Amazon ECS. The webserver is configured to present TLS certificates to incoming requests, receive encrypted TLS traffic. The web server is part of a ELB v2 Target Group. There are two hops in this flow:

  • Customer iPhone/laptop/whatever connects to Amazon ALB. You can upload a certificate to Amazon to present to the customer. The customer's iPhone/browser/whatever will (hopefully) verify that certificate before sending requests to the ALB.

  • The ALB forwards the request to your webserver. ALB will look up the right Listener for the request, and then forward it to a ELB v2 Target Group. You can configure the Target Group to receive requests over HTTP or HTTPS.

  • If you choose HTTPS, the ALB will establish a connection and request a certificate from a random host in the Target Group. It will not validate that certificate; it will just send the traffic.

Here is the configuration for a Target Group. There's no check box for "validate HTTPS traffic from internal service."

The entire point of HTTPS is to encrypt traffic. Otherwise, a random person snooping on the network could present a weak certificate and send back whatever data it wants. We learned in 2014 that the NSA was doing this, between and inside data centers and at key points in the US.

It's unacceptable for a major Internet service in 2018 to blindly accept certificates presented by an internal service without validating them.

For the moment, I suggest using the Network Load Balancer type, which forwards the raw TCP traffic to your machine. You don't get any of the nice features of an ALB, but at least you will have the ability to reject raw traffic. If you know of other providers that offer load balancers with TLS certificate validation, please send me an email.

Update: Please read the reply from AWS.

Liked what you read? I am available for hire.

Otto LLC Assignment for Benefit of Creditors (bankruptcy)

Posted on

Twelve days before Christmas 2017, Otto LLC, a company that wanted to produce smart locks, shut down with little notice. Employees were given no severance and contractors were left with tens or hundreds of thousands of dollars in unpaid invoices. This incident was widely covered in the press, and one board member has since quit the venture capital industry.

Otto has sold all of their assets to a company that specializes in liquidation and is now going through an "assignment for benefit of creditors."

I am not a lawyer and this is not legal advice, but that said, here is how this works at a high level. Everyone submits their claims to the Assignee. These claims have different levels of priority; people who lent money to the company have one priority, people who are owed money for services have a different priority, people who own shares in the company have a different priority. Generally, all of the claims at one priority are paid out before proceeding to the next class.

The Assignee is required to post notice in a newspaper, that's it - they don't have a requirement to notify you personally. I found the address for submitting claims, though. If Otto owes you money, go here to submit your claim: https://fileaclaim.info/Otto-ABC. You have until November 2, 2018.

Yes, that is the real URL.

I wish you the best of luck, however, I fear that there is not very much money left to pay claims, so you might not get very much. Still, a little bit is better than nothing.

If there are any questions I can answer for you, or if there's anything I can help you do to find a new position, please get in touch.

Liked what you read? I am available for hire.

Opportunity for more affordable housing in Belmont

Posted on

Belmont is finalizing plans to build affordable housing on two parcels it owns across from the Caltrain station along El Camino Real. The proposal from the developer, LINC Housing, would construct around 20 large-family affordable units (2 and 3 bedrooms) and 20 senior housing units (one bedroom).

Mockup of proposed BMR project

Belmont reduced the parking requirements for new construction last year as part of the Village Specific Plan update. The large-family units would require 1.5 parking spaces each, and the senior units would require 1 parking space each, for a total of around 50 units. It is likely this parking would have to go underground. At a cost of $60,000 per below-ground parking space, it would cost around $3 million to build all of the required parking below ground, or about 15% of the total cost of the project.

Street layout of proposed BMR project

Changing transportation technology and services are already starting to cut into parking demand in some places (see this story from San Diego).

In the event that the below-ground parking becomes redundant, it will be difficult to repurpose for a more productive use. You can’t build apartments underground, and it’s unattractive space for businesses. So it would be prudent for the city to consider alternatives to building more parking than it may need.

Quite a lot of parking already exists in close proximity to the proposed sites, north of the Caltrain station across the street. A June 2017 Caltrain parking analysis indicated that just 2% of the spaces were occupied on the given weekday.

Caltrain utilization

I snapped these photos of the lot at 1pm in April 2018.

Empty lot across the street

What if Belmont were to enter into a parking sharing agreement with Caltrain? Caltrain could issue permits to residents of the proposed new development. Even 50 permits would not result in the lot being full, and the extra cars would mostly use the lot at night, when commuters are gone and the Caltrain lot is empty.

There is a precedent here. Caltrain and the City of Mountain View developed a shared parking agreement with a townhouse developer near the San Antonio station, as part of the San Antonio Station Precise Plan.

San Antonio agreement

Using the parking lot across the street would free up about $3 million. What could Belmont do with that money? For $200,000, the city could build a HAWK pedestrian crossing at Hill St, shaving 4 minutes off the walk to the train station and the northbound ECR bus stop. This crossing is proposed in the Belmont Pedestrian Master Plan.

HAWK installation at Hill Street

Further, lifting the on-site parking requirement might make it feasible to develop Site C (780 El Camino Real), which is owned by the city. LINC Housing's proposal did not include development for this site. A sloped, funky-shaped lot would be a lot easier to develop if it didn't require a parking ramp and additional driveway.

Site C

What do you think? Send your thoughts to the Belmont City Council (CityCouncil@belmont.gov), as well as the city’s project lead, Jennifer Rose (jrose@belmont.gov) and community director, Carlos de Melo (cdemelo@belmont.gov).

Liked what you read? I am available for hire.

Housing by the numbers

Posted on

There's a pretty famous software engineering slide called "Latency Numbers Every Programmer Should Know." I thought I would write an equivalent for housing construction, since a lot of these statistics can be hard to estimate. The goal is to make it easy to do a ballpark estimate of the impacts of a particular proposal. For example, Emeryville has a $50 million affordable housing bond on the ballot, and California has a $4 billion housing bond. How many housing units would each allow to be built?

  • Housing units California must build every year to meet population/household growth: 180,000, according to the California Department of Housing and Community Development. California currently has 13 million homes; spread evenly, each city would need to grow their housing stock by about 1.4% per year.

  • Housing units California has built over the past decade: About 64,000 per year.

Chart of housing production per year

  • Cost to build a single housing unit: This depends on a few things:

    • cost to buy the underlying land

    • how much you pay construction workers. Labor groups push for "prevailing wage," a living wage established by the state government. The prevailing wage in Los Angeles is around $43 per hour, while the median wage is around $25. If projects don't pay prevailing wage, unions will sometimes file lawsuits alleging that projects violate CEQA - an environmental quality law. The lawsuit will then be dropped if the project agrees to pay prevailing wage.

    • environmental regulations in new construction, anything from gray water / solar requirements to CEQA.

    • the amount of time it takes to get through the application process, and the amount of local opposition. In San Francisco it is not unusual for an application to take 10 years from end to end.

At the low end, you might be able to build an apartment building for $250,000 per unit. At the high end (San Francisco) it might be $600,000 or $700,000 per unit.

  • Square feet per unit: A studio might be as low as 300 square feet. A two or three bedroom apartment might be 1200 square feet. A single family home might be 2000 square feet. A mansion might be 5000 square feet.

  • Square feet per employee, for office space: Anywhere from 120 to 200 square feet per person. This disparity means you can build office space for more workers on the same lot than you can build space for people to live.

    For example, if you hear that Brisbane wants to add 2,200 housing units and 4 million square feet of office space, you can assume at the high end that 4 million square feet is 33,000 commuters, and at the low end that it's 20,000 commuters.

  • Construction cost per square foot: In the Bay Area, $400 per square foot. Modular housing may be able to reduce these costs, by building housing offsite in more friendly environments, then shipping fully constructed units to the site.

  • Cost for the State of California to build 180,000 units per year by itself: At $250,000 per unit, $45 billion; at $400,000 per unit, $72 billion. The state budget is $190 billion this year. AB 71 would have raised $300 million for housing by repealing the state mortgage interest deduction on second homes. This measure would have raised about 0.4% of the required yearly amount, but failed to get a vote in the Legislature last fall.

  • Parking required per housing unit: Anywhere from 0 to 2 spaces required per unit. In some parts of San Francisco you are not required to build parking spaces at all. Belmont, where I live, recently revised its parking requirement per unit down from 2 spaces per unit to 0.5 per studio, 1 per 1 bedroom, 1.5 per unit above that.

  • Cost to build a parking space: Above ground, about $22,000 to $27,000 per space.

  • Cost to build an underground parking space: About $60,000 per space.

  • Cost to get a mortgage: Anywhere from 20-30% of the purchase price for a down payment, and then a loan for the remaining amount with an interest rate between 4% and 7%, depending on your credit score.

  • Cost of the median home: $1.5 million in San Francisco, $1.3 million in San Mateo County. Starter homes/condos are at least $700k-800k.

  • Property taxes: About one percent of the purchase price of the home; if you pay $1 million for a home, you pay $10,000 per year in property taxes. Thanks to Prop 13, your property tax can increase by a maximum of 2% per year. (Home values have increased by much more than 2% per year.) This leads to absurd situations like side-by-side homes, one of which is paying $2000 per year in property taxes and one of which is paying $13,000 per year in taxes.

  • Rate of return for a multifamily apartment developer: Anywhere from 5% to 10% to be able to qualify for a loan from a bank to build the structure. Of course, in downturns like 2008, developers might lose all of their money. The historical rate of return for the Dow Jones Industrial Average is 7.75%.

Thanks to Scott Feeney for reviewing a draft of this post. What other statistics are you curious about? Send me an email: kev@inburke.com

Liked what you read? I am available for hire.

Profile Anything in Any Language in Under a Minute

Posted on

Want to know how to get rough profiling of any tool written in any language, whether you control the source or not? Keep reading.

For example, here's the output you get when you compile Go from source code. Each of these lines prints out a few seconds apart. How would you go about getting timings for each individual step, or how would you notice if one step was suddenly executing much more slowly than it did three days ago?

$ GOROOT_BOOTSTRAP=~/go1.10 ./make.bash
Building Go cmd/dist using /Users/kevin/go1.10.
Building Go toolchain1 using /Users/kevin/go1.10.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
---
Installed Go for darwin/amd64 in /Users/kevin/go
Installed commands in /Users/kevin/go/bin

There are a few different methods you can use to get a sense for this, but I wanted to share my favorite one. Don't reach for a profiler, don't try to set a global variable, or do start/stop timing in code. Don't even start figuring out how to configure a logger to print timestamps, and use a log output format.

Then when you've done all this, rerun the command and pass it through a command that attaches timestamps to every output line. For example, I have a tool called tss (based on ts in the moreutils package) that does this. Pipe your command to tss and you'll get output like this:

$ GOROOT_BOOTSTRAP=~/go1.10 ./make.bash | tss
     11ms         Building Go cmd/dist using /Users/kevin/go1.10.
    690ms   679ms Building Go toolchain1 using /Users/kevin/go1.10.
  11.435s 10.745s Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
  18.483s  7.048s Building Go toolchain2 using go_bootstrap and Go toolchain1.
  40.788s 22.305s Building Go toolchain3 using go_bootstrap and Go toolchain2.
 1m0.112s 19.324s Building packages and commands for darwin/amd64.
1m19.053s 18.942s ---
1m19.053s      0s Installed Go for darwin/amd64 in /Users/kevin/go
1m19.053s      0s Installed commands in /Users/kevin/go/bin

The first column is the amount of time that has elapsed since the tool was invoked. The second column is the amount of time that has elapsed since the previous line was printed.

The good thing about this technique is you can use it in pretty much any situation - you don't have to know anything about the code besides:

  • how to invoke it
  • a rough idea of which code paths get hit
  • how to print things to stdout

Just start adding print lines around interesting bits of code.

For example, you might have this in your test suite in Ruby to empty the database. How much time does this add to every test in your suite? Put print lines before and after to find out.

config.after(:each) do
  puts "database clean start"
  DatabaseCleaner.clean
  puts "database clean end"
end

Similarly, if you wanted to find out why it takes so long for your test suite to start running or your web framework to boot, you can annotate your spec_helper or bootstrap file, the top of the test file, before and after require lines, and the start and end of the first test in the suite. You can even drill down into third party libraries, just find the source file on disk and start adding print statements.

You can also use it with strace! strace will show you a list of the syscalls being opened by your program. Passing the output through tss can show you which syscall activity is taking a lot of time, or when your program is doing a lot of stuff without going to the disk.

Caveats

If you pipe program a to program b, and program a exits with a non-zero return code, Bash will by default report a return code of zero for the entire operation. This means if you are piping output to tss or ts you may accidentally change a failing program to a passing one. Use set -o pipefail in Bash scripts to ensure that Bash will return a non-zero return code if any part of a pipe operation fails. Or add this to a Makefile:

SHELL = /bin/bash -o pipefail

Second, the program being profiled may not flush output to stdout, that is, it may print something but it may not appear on the screen at the same time it was printed. If the timings on screen don't seem to match up with your intuition, check that the program is flushing print statements to the stdout file descriptor after printing them.

Liked what you read? I am available for hire.

Where is everyone going to live?

Posted on

In the middle of a housing crisis, cities up and down the Peninsula are moving forward with plans to add a ton of new office space and no new housing. These plans will add many more new commuters, without any corresponding housing for them.

  • Last year the Apple Park launched in Cupertino. 2.8 million sq ft of office space (room for 18,000 workers), no new housing.

  • Brisbane just put a plan for the Baylands on the 2018 ballot. 4 million sq ft of office space (room for 26,000 workers), 2200 housing units.

  • South San Francisco is moving forward with Oyster Point development. 1.5 million sq ft of office space (room for 10,000 workers), 1200 housing units.

  • San Francisco is moving forward with the Central SOMA Plan. 6 million sq ft of office space (room for 40,000 workers), only 7000 housing units.

  • Facebook wants to add 35,000 employees in Menlo Park, with only 1500 housing units to match.

Where are all of those commuters supposed to live? No one is planning on adding housing to match the new office space being added.

We have seen this story play out already. Those rich new workers are going to want apartments near their offices, so they'll bid up the price of existing housing. This will lead to gentrification and displacement. Everyone who can't afford to do that will commute from Pleasanton, Gilroy and Stockton. This will clog 101 and the San Mateo Bridge, leading to even worse traffic.

It cracks me up when folks say "Why doesn't Google just build their next office in Bakersfield?" Your own City Councils are rolling out the red carpet to host their next office, so they can get that sweet revenue, and don't care/don't want to add any housing to match. This is what "local control" has gotten us and the results have been ruinous.

My brother and sister have already left the Bay Area. My best friend grew up 300 yards from me and now is raising his family in Savannah, GA. I'm worried I'm going to be next. All of these new developments are the world's slowest moving steamroller, driving up rent and housing prices. $3m for a home and $4,000 median rents are in our near future.

How can we turn these traffic and displacement bombs around?

  • Call your Senator and Assemblymember to support SB 827, which could actually add enough new housing near transit to reverse our traffic problems and slow the explosive growth of rents.

  • Support new housing in your town. The State is adding bonuses for adding new housing units, but developers are choosing not to use them, for fear of community backlash. Tonight, for example, there's a meeting about 935 homes at Concar Drive at San Mateo City Hall at 7pm, and another at Millbrae City Hall about the Serra Station project adjacent to Millbrae BART. These could use voices in support.

  • If you're someone who does not benefit from a fixed rate mortgage and Prop 13, or if you are and you're worried about where your kids are going to live, consider running for City Council or applying for Planning Commission in your town. Sign up for mailing lists to get notified about new events happening: I run one (kev@inburke.com) or yimbyaction.org/sanmateo.

Liked what you read? I am available for hire.