Nabble - PostgreSQL - San Francisco

12/08 SFPUG meeting, "Operator Exclusion Constraints," video now available

2009-12-23T12:24:34Z

The video archive for the 12/08 SFPUG meeting, "Operator Exclusion
Constraints," is now available:

http://thebuild.com/blog/2009/12/23/sfpug-operator-exclusion-constraints/

--
-- Christophe Pettus
xof@...

Re: [GENERAL] 11/10 SFPUG meeting, "Continuent Tungsten with PostgreSQL," video now available

2009-12-22T22:25:20Z

Looks great. Thanks again! Robert

On 12/22/09 10:17 PM PST, "Christophe Pettus" <xof@...> wrote:

> Ah, you think you have all the links in an entry corrected... :) It's fixed
> now, thanks!
>
> On Dec 22, 2009, at 10:14 PM, Robert Hodges wrote:
>
>> Hi Christophe,
>>
>> Thanks for posting the video, but the linked video appears to be Josh Berkus
>> lecturing on GUCS, which I believe was the previous month.
>>
>> BTW, since doing the Tungsten talk in early November we have also
>> implemented support for clusters using Londiste. We¹ll do some talks about
>> it in the new year.
>>
>> Cheers, Robert Hodges
>> (aka one of the Tungsten guys)
>>
>

Re: [GENERAL] 11/10 SFPUG meeting, "Continuent Tungsten with PostgreSQL," video now available

2009-12-22T22:17:45Z

Ah, you think you have all the links in an entry corrected... :) It's fixed now, thanks!

On Dec 22, 2009, at 10:14 PM, Robert Hodges wrote:

Hi Christophe,

Thanks for posting the video, but the linked video appears to be Josh Berkus lecturing on GUCS, which I believe was the previous month.

BTW, since doing the Tungsten talk in early November we have also implemented support for clusters using Londiste. We’ll do some talks about it in the new year.

Cheers, Robert Hodges
(aka one of the Tungsten guys)

On 12/22/09 10:08 PM PST, "Christophe Pettus" <xof@...> wrote:

The video archive for the 11/10 SFPUG meeting, "Continuent Tungsten
with PostgreSQL," is now available:

http://thebuild.com/blog/2009/12/22/sfpug-continuent-tungsten-with-postgresql/
--
-- Christophe Pettus
xof@...

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

-- Christophe Pettus

xof@...

Re: [GENERAL] 11/10 SFPUG meeting, "Continuent Tungsten with PostgreSQL," video now available

2009-12-22T22:14:05Z

Re: [GENERAL] 11/10 SFPUG meeting, "Continuent Tungsten with PostgreSQL," video now available Hi Christophe,

Thanks for posting the video, but the linked video appears to be Josh Berkus lecturing on GUCS, which I believe was the previous month.

BTW, since doing the Tungsten talk in early November we have also implemented support for clusters using Londiste. We’ll do some talks about it in the new year.

Cheers, Robert Hodges
(aka one of the Tungsten guys)

On 12/22/09 10:08 PM PST, "Christophe Pettus" <xof@...> wrote:

The video archive for the 11/10 SFPUG meeting, "Continuent Tungsten
with PostgreSQL," is now available:

http://thebuild.com/blog/2009/12/22/sfpug-continuent-tungsten-with-postgresql/
--
-- Christophe Pettus
xof@...

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

11/10 SFPUG meeting, "Continuent Tungsten with PostgreSQL," video now available

2009-12-22T22:08:06Z

The video archive for the 11/10 SFPUG meeting, "Continuent Tungsten
with PostgreSQL," is now available:

http://thebuild.com/blog/2009/12/22/sfpug-continuent-tungsten-with-postgresql/
--
-- Christophe Pettus
xof@...

[job] DBA position in San Mateo

2009-12-16T15:42:38Z

If you are interested in this position, please contact:

jigar@...

Do NOT contact me, or the mailing list.

The contact says he also has 2 sysadmin positions.

===============

PostgreSQL Database Administrator

Your responsibilities will focus on service delivery: designing,
implementing, and maintaining a best-in-class infrastructure with strict
availability and performance requirements. You'll be expected to work
across functional boundaries, contribute to process development, and
participate in an on-call rotation. The most important job requirement
is the ability, experience, and desire to provide a computationally
intensive service, non-stop. Your troubleshooting skills should be
top-notch, and your communication skills should be strong. We'd like to
see experience in at least a few key areas: performance analysis,
advanced storage systems, network engineering, and
infrastructure/support systems.

Responsibilities:

* Rotating 24x7 On-call support.
* Develop high performance relational databases to store content,
user personalization information, and billing/tracking data, etc.
Develop and implement maintenance and administration procedures for the
database systems.
* Ensures that backups are performed regularly on all databases.
* Work closely with software engineers to develop and maintain
stored procedures, compiled SQL, etc. Ensure that data access methods,
database kernel patches, APIs, etc. are kept current.
* Develop a system for gathering database performance metrics. Using
these metrics, develop a plan to ensure that database systems are
correctly configured for optimal performance. Manage and/or participate
in implementation of database system tuning, to ensure optimal performance.
* Ensure that database systems are adequately monitored. Develop and
implement a plan to ensure maximum reliability and availability of
database systems. Proactively monitor database system status to
determine early warning signs of problems.
* Author and maintain scripts to perform various aspects of
administration. The DBA will document these scripts and maintain them in
the source repository.

Requirements:

* 5 years experience as a DBA.
* 3 years experience of Postgres administration and tuning in a
Linux environment.
* Must have strong knowledge of data relationships, data normalization.
* SQL, Perl, python, ruby, bash, C or other scripting experience.
* Understanding of LVM, SAN and RAID.
* Cacti, Nagios, or other monitoring and metrics tools.
* Ability to learn quickly and adapt to a fast paced growing and
evolving service delivery system.
* Ability to manage projects through their entire life cycle.
* Excellent communication skills.

Key Strengths:

* Able to troubleshoot and resolve network, application, and systems
issues expediently.
* Be flexible to work rotating shifts in a 24 x 7 environment.
* Possess excellent verbal and written communication skills.
* Occasional visits to our local data center will be needed, so
transportation is a necessity.

Re: Recommended/Not Recommended Hosts?

2009-12-15T15:59:18Z

> Seek performance on the non-EBS default drive is poor -- but if I'm
> reading the bonnie++ output right, the EBS volume otherwise wasn't
> really faster - and perhaps even slower than the default drive for block
> reads and writes. Also if I read the bonnie++ output right, strangely,
> block reads seem much slower with a 7-EBS-volume RAID array than with a
> single EBS block. RAID helped the writes significantly, though.

Yeah, that's fairly odd. Is it possible that the CPU overhead of the SW
RAID is eating a lot of your available CPU?

It would also be great to run Bonnie 1.95 and look at lag times.

--Josh

Submit talks to SCALE!

2009-12-13T15:28:32Z

All,

the SCALE call for papers is still open. Submit your talks, now!

www.socallinuxexpo.com

I will be driving down to talk and to staff the PostgreSQL booth at the
conference, so I'll have room for 2 passengers.

--Josh Berkus

Re: Recommended/Not Recommended Hosts?

2009-12-11T17:14:06Z

Jason DiCioccio wrote:
> On Fri, Dec 11, 2009 at 15:48, Sailesh Krishnamurthy <sailesh@...> wrote:
>> Has anyone used EBS on EC2 ?
>> I/O is supposed to be much better on it
>
> My testing has shown it to still be pretty awful. Although Amazon
> does suggest doing software RAID on multiple EBS volumes. I haven't
> tried that yet.

The one thing I love about EC2 is that it's incredibly easy and
cheap to test such things. I just tried it for the cost of about $0.20.

(the script I used, and output from bonnie++ below)

Seek performance on the non-EBS default drive is poor -- but if I'm
reading the bonnie++ output right, the EBS volume otherwise wasn't
really faster - and perhaps even slower than the default drive for block
reads and writes. Also if I read the bonnie++ output right, strangely,
block reads seem much slower with a 7-EBS-volume RAID array than with a
single EBS block. RAID helped the writes significantly, though.

Does that make sense? Can anyone help me interpret that output?

All this was on a small instance (which end up with low I/O and CPU priority),
so I guess it's quite possible this test was dominated with people running more
expensive instances stealing the CPU and I/O resources.

Here's the script I just used after attaching 7 EBS volumes to one of my
instances:
================================================================================
sudo mdadm --create /dev/md0 --level=0 --raid-devices=7 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdb4 /dev/sdb5 /dev/sdb6 /dev/sdb7
sudo mkfs.ext3 /dev/md0
sudo mkdir /mnt/md0
sudo tune2fs -o journal_data_writeback /dev/md0
sudo mount /dev/md0 /mnt/md0
sudo chmod a+w /mnt/md0
cd /mnt/md0
bonnie++ -d /mnt/md0
================================================================================

And here's the output:

================================================================================
= 1 local disk on EC2 small instance
===============================================================================
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ars.forensicl 3408M 21139 35 65132 11 31224 2 24915 19 70138 1 189.1 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
ars.forensiclogic.com,3408M,21139,35,65132,11,31224,2,24915,19,70138,1,189.1,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

================================================================================
= 1 EBS disk on EC2 small instance
===============================================================================

Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ars.forensicl 3408M 22239 34 40111 5 17178 0 23539 17 51379 0 570.1 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
ars.forensiclogic.com,3408M,22239,34,40111,5,17178,0,23539,17,51379,0,570.1,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

================================================================================
= RAID0 of 7 EBS disks on EC2 small instance
===============================================================================

Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ars.forensicl 3408M 20884 34 115135 24 17586 1 15472 20 23198 0 5659 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
ars.forensiclogic.com,3408M,20884,34,115135,24,17586,1,15472,20,23198,0,5659.1,1,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

Re: Recommended/Not Recommended Hosts?

2009-12-11T16:21:35Z

On Fri, Dec 11, 2009 at 15:48, Sailesh Krishnamurthy <sailesh@...> wrote:
> Has anyone used EBS on EC2 ?
>
> I/O is supposed to be much better on it
>
> Sailesh
>

My testing has shown it to still be pretty awful. Although Amazon
does suggest doing software RAID on multiple EBS volumes. I haven't
tried that yet.

Re: Recommended/Not Recommended Hosts?

2009-12-11T15:48:36Z

Has anyone used EBS on EC2 ?

I/O is supposed to be much better on it

Sailesh

On 12/10/09, Josh Livni <josh@...> wrote:

> Cool, Thanks for the detailed response. I've certainly not done a ton of
> research comparing different VPS solutions myself, but I was under the
> (quite possible mis-impression) that at EC2 the underlying hardware was not
> 1:1 related to your performance (eg you get a set amount of cpu throughput,
> and if they had older hardware underneath, then you'd just get more of it).
> I also did not realize other users could steal cycles from you like happens
> on most other VPS offerings. I haven't seen that much documentation to base
> any of these assumptions on, of course, so it's good to hear your
> perspective.
>
> Some posts, such as http://journal.uggedal.com/vps-performance-comparison,
> seem to imply different conclusions (not that he's suggesting EC2 is a good
> deal, but for different reasons than cpu stealing), and I'd love to see
> similar posts on the topic: I'd be happy to switch to something else if I
> felt I was going to getting a much better deal (I really do like the
> integrated EBS/S3/Cloudfront options tho for the type of projects I
> generally work on).
>
> Cheers,
>
> -Josh
>
>
> On Thu, Dec 10, 2009 at 11:20 AM, Josh Berkus <josh@...> wrote:
>
>> Josh,
>>
>> > Pure curiosity on my part here ... I use EC2 a bit, tho not as much as
>> > the serious users. A few large and small instances on all the time, and
>> > I boot up new ones for shorter periods all the time. First - I've
>> > never had any issue getting my instances fulfilled right away (I always
>> > use EAST-C, but perhaps other datacenters are generally more full, or
>> > you are trying to boot up many tens of servers at once?).
>>
>> Yeah, the two issues I've had are (a) requisitioning high-end instances
>> (like 32G/16core instances) and (b) allocating a lot at once. Sometimes
>> instances just "aren't available" and there's no way to find out when
>> they will be available.
>>
>> > Also, when you say they are slow, do you mean in terms of $/cycle,
>> > or you wish you had burst access to other users unused cycles like on
>> > some other vps offerings? something else?
>>
>> I mean that if you have an 8core/16GB instance, the actual processing
>> throughput you get is about 1/6 to 1/4 that of a new HP DL380 machine
>> with 8cores and 16GB. So you really need 4x as many EC2 instances to
>> match bare metal. Partly this is due to CPU-stealing, and partly to
>> erratic and lag-prone I/O, and partly to the fact that a lot of machines
>> in the EC2 pool are 4 years old.
>>
>> > I like the bundle of offerings that AWS provides (EBS, especially), and
>> > I've personally had great experience w/them (fwiw I've also had great
>> > experience w/slicehost) -- but if I am getting missing out on how
>> > they're screwing me, for example by stealing my CPU, I'd definitely love
>> > to learn more.
>>
>> On EC2, other VMs on the same hardware are permitted to "steal" portions
>> of the CPU which are allocated to you. So at any given time, you may
>> have as little as 50% of the CPUs you're being billed for. And, when
>> CPU availability is fluctuating up and down (as it does on EC2), real
>> throughput tends to be based on the slowest second rather than peak
>> availablity. Most Linux apps, especially databases, do quite poorly
>> with erratic resource availability.
>>
>> --Josh
>>
>

--
Sent from my mobile device

Cheers
Sailesh

Re: Recommended/Not Recommended Hosts?

2009-12-11T15:18:41Z

On Dec 11, 2009, at 2:49 PM, Josh Berkus wrote:

> All,
>
> I just found www.zill.net, which appears to be dedicated PostgreSQL
> hosting. Anyone have experience with them?

Not I. They look a bit zombie, though - their support page provides docs for Postgresql 7.2.1 and some OS9 scp client.

Cheers,
Steve

Re: Recommended/Not Recommended Hosts?

2009-12-11T14:49:27Z

All,

I just found www.zill.net, which appears to be dedicated PostgreSQL
hosting. Anyone have experience with them?

--Josh Berkus

Re: Recommended/Not Recommended Hosts?

2009-12-10T13:55:45Z

On Dec 10, 2009, at 10:43 AM, Josh Berkus wrote:

> All,
>
> What hosts, both virtual hosts and colos, do you recommend for
> PostgreSQL-based applications, and which ones have you had bad
> experiences with? If we get a list together, we'll put it up somewhere
> community.
>
> For my part, I've had reasonably good experiences with:
>
> -- Rackspace rental servers (provided you're OK with Dell hardware)
> -- Layer42 for low-end colo, but watch your bandwith, they won't.
> -- Joyent for online data warehousing (if you can deal with Solaris)

I've been with Integra Telecom (nee ELI) for quite a few years. If you're
big enough that you need somewhere between a full cabinet and
a small cage, but are happy with a default route then they're well worth
a look. Solid provider, and very helpful NOC / support (even when
dealing with someone dumb enough to have let his switch
autonegotiate...). They're in Santa Clara.

I've also had a couple of U of space at Hurricane Electric in Fremont
forever, mostly as a backup location, and it's been pretty solid. It's been
years since I've actually been there, so I don't even remember what
the physical facility is like. (World+Dog resells space there, including
http://www.asaservers.com/ who'll sell you a box and host it at HE
for you).

Cheers,
Steve

Re: Recommended/Not Recommended Hosts?

2009-12-10T12:05:49Z

Fred Moyer wrote:
> On Thu, Dec 10, 2009 at 10:43 AM, Josh Berkus <josh@...> wrote:
>> For my part, I've had reasonably good experiences with:
>>
>> -- Layer42 for low-end colo, but watch your bandwith, they won't.
>
> At the risk of giving away one of my best kept secrets, Layer42 is
> simply the most outstanding and professional colocation provider I
> have ever had the pleasure to work with. Their network is FAST and
> unbelievably reliable. Steve and his team are of the highest caliber.

I've been at Layer 42 for five years so I do believe they provide a good
service for the price. It's worth noting however they have suffered at
least two full power outages over the past couple of years. While they
have been brief (~seconds), they cause hardware to power cycle and we've
had hardware failures each time. The most recent outage was in September.

Brian

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:59:39Z

> Just to clarify, Joyent does this too, no? Or is it just that the
> Solaris scheduler handles the prioritization of tasks/resources
> better? I haven't yet had the chance to use Joyent, but I thought
> this was one of their selling points.

It depends on the size of your instance. If you're using a whole CPU
(not core, CPU) Joyent can dedicate that CPU to you and nobody else can
touch it. One of the advantages of Solaris over Xen.

--Josh

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:56:58Z

Josh Berkus wrote:
> What hosts, both virtual hosts and colos, do you recommend for
> PostgreSQL-based applications...
> For my part, I've had reasonably good experiences with:
> -- Rackspace rental servers (provided you're OK with Dell hardware)
> -- Layer42 for low-end colo, but watch your bandwith, they won't.
> -- Joyent for online data warehousing (if you can deal with Solaris)

For the extremely low end of dedicated servers, I'm very happy with
Server4You ( http://www.server4you.com ) and have been running
postgres-backed hobby sites on a few of their machines for
years. $29/month for a tiny cheap dedicated box is great for
small cheap clients starting out, and it's easy to upgrade their
plans to their slightly larger systems. They have nothing high-end,
though.

For the high-end systems we run our own servers at 365 Main in
SF (where we have mixed experiences - including a painful unplanned
downtime but it was rather affordable) and a Sungard facility in
Texas (which was much better, but not at all cheap).

> I've had bad experiences with:
>
> -- Amazon EC2: uptime and availability are great, but the servers are
> sloooooooow and fulfillment of new instances is unreliable. Also,
> CPU-stealing.

Curious which instance type you had those experiences with.

Amazon provides (expensive) solutions to help address each of
those complaints, at least to some degree.

* Their "High CPU" Instances (and I think even moreso their poorly
named "High Memory" instances) have vastly improved performance
over their sloooooow normal instances. At least an order of
magnitude when I tried, and 26 times as fast if you believe
their docs[1]. Those things go for $2.40 instead of $0.085
per hour, though [2]. The I/O performance of their small
instances are horribly too; and again much improved in their
expensive ones. You can also make RAID 0 arrays of their EBS
volumes that have higher performance than one.

I haven't noticed CPU stealing on the high-CPU instance I ran.

* Fulfillment of new instances is supposed to be handled by
their "Reserved Instances" feature. You make a one-time
payment of between 350 and 2800 dollars they're supposed
to keep it available for you for 1-3 years.

I like Amazon for servers and clusters don't need to run 24x7.
For example ones that only need to be up during business hours
or peek traffic periods. Also for short-lived servers - for
example I'll be setting up an experimental hadoopdb cluster
there. For servers running 24x7 for 30 days or more, it
seems to me other solutions are more cost effective.

[1] http://aws.amazon.com/ec2/instance-types/
[2] http://aws.amazon.com/ec2/#pricing

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:55:11Z

On Thu, Dec 10, 2009 at 11:20, Josh Berkus <josh@...> wrote:
> On EC2, other VMs on the same hardware are permitted to "steal" portions
> of the CPU which are allocated to you. So at any given time, you may
> have as little as 50% of the CPUs you're being billed for. And, when
> CPU availability is fluctuating up and down (as it does on EC2), real
> throughput tends to be based on the slowest second rather than peak
> availablity. Most Linux apps, especially databases, do quite poorly
> with erratic resource availability.

Just to clarify, Joyent does this too, no? Or is it just that the
Solaris scheduler handles the prioritization of tasks/resources
better? I haven't yet had the chance to use Joyent, but I thought
this was one of their selling points.

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:54:00Z

> Pure curiosity on my part here ... I use EC2 a bit, tho not as much as
> the serious users. A few large and small instances on all the time, and
> I boot up new ones for shorter periods all the time. First - I've
> never had any issue getting my instances fulfilled right away (I always
> use EAST-C, but perhaps other datacenters are generally more full, or
> you are trying to boot up many tens of servers at once?).

Yeah, the two issues I've had are (a) requisitioning high-end instances
(like 32G/16core instances) and (b) allocating a lot at once. Sometimes
instances just "aren't available" and there's no way to find out when
they will be available.

> Also, when you say they are slow, do you mean in terms of $/cycle,
> or you wish you had burst access to other users unused cycles like on
> some other vps offerings? something else?

I mean that if you have an 8core/16GB instance, the actual processing
throughput you get is about 1/6 to 1/4 that of a new HP DL380 machine
with 8cores and 16GB. So you really need 4x as many EC2 instances to
match bare metal. Partly this is due to CPU-stealing, and partly to
erratic and lag-prone I/O, and partly to the fact that a lot of machines
in the EC2 pool are 4 years old.

> I like the bundle of offerings that AWS provides (EBS, especially), and
> I've personally had great experience w/them (fwiw I've also had great
> experience w/slicehost) -- but if I am getting missing out on how
> they're screwing me, for example by stealing my CPU, I'd definitely love
> to learn more.

On EC2, other VMs on the same hardware are permitted to "steal" portions
of the CPU which are allocated to you. So at any given time, you may
have as little as 50% of the CPUs you're being billed for. And, when
CPU availability is fluctuating up and down (as it does on EC2), real
throughput tends to be based on the slowest second rather than peak
availablity. Most Linux apps, especially databases, do quite poorly
with erratic resource availability.

--Josh

For a dev environment running a small database, we've had a great experience with Rackspace cloud servers. The price point for server instances is around $10 per 256 MB which is the low end for cloud services.

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:52:22Z

On Thu, Dec 10, 2009 at 10:43, Josh Berkus <josh@...> wrote:
> All,
>
> What hosts, both virtual hosts and colos, do you recommend for
> PostgreSQL-based applications, and which ones have you had bad
> experiences with? If we get a list together, we'll put it up somewhere
> community.
>

From the colo perspective:

If your app is bandwidth-hungry, I recommend Equinix or some other
carrier neutral facility where you can get good bandwidth prices.
Space will be more expensive, but if you're using a lot of bandwidth,
it should be cheaper and more reliable overall.

Now for carriers in such a facility (I generally run BGP with multiple
carriers):

*) Cogent - I've had good luck with them overall, and they're cheap.
*) Level(3) - No issues with them either. They also offer IPv6 now I
believe. NOC is kind of a pain though to get through to though.
*) Internap - They're fine, but I don't think they're worth their
price premium. This is all anecdotal, but I've had more reliability
issues with them than I've had with Cogent. So YMMV. If you're going
to only have *one* carrier though, this might be a good choice as they
have transit (not peering) agreements with everyone that should make
them immune to the various political peering disputes that have
happened over the years.

There are a ton of other carriers that I don't have recent experience
with, so ask around.

Usually, I'll setup 2-3 carriers, then give Cogent highest preference
for all traffic that doesn't terminate on one of the other carriers.
Otherwise the other carrier wins. I find that it's a good way to go
for bandwidth cost savings, but now I'm getting into more of a network
engineering discussion. Dedicated servers/cloud stuff can help you
avoid this infrastructure stuff, but at the cost of much higher MRCs
when you get above very small deployment sizes. But if you're at the
stage where you're wondering which host to use, that's probably the
size you're at.

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:45:49Z

I signed up for a year of hosting with host1plus listed as a vendor here: http://www.postgresql.org/support/professional_hosting_europe

Based out of the UK, but they have servers in the US. The deal looked pretty good. But the hosting went down for days at a time with with no response and site off line. I just took my sight and moved it to another vendor.

I was not using postgreSQL in my hosting solution, but I had chosen them so I could at some point in the future. I can give them a very strong not recommended.

-Jeff

Josh Berkus wrote:

All,

What hosts, both virtual hosts and colos, do you recommend for
PostgreSQL-based applications, and which ones have you had bad
experiences with?   If we get a list together, we'll put it up somewhere
community.

For my part, I've had reasonably good experiences with:

-- Rackspace rental servers (provided you're OK with Dell hardware)
-- Layer42 for low-end colo, but watch your bandwith, they won't.
-- Joyent for online data warehousing (if you can deal with Solaris)

I've had bad experiences with:

-- GoGrid Cloud Servers ... I/O is very very bad, so your database
better fit in RAM, and they're limited to 8GB max.  On the other hand,
no downtime whatsoever. (in GG's defense, they recommend colo for databases)

-- Amazon EC2: uptime and availability are great, but the servers are
sloooooooow and fulfillment of new instances is unreliable.  Also,
CPU-stealing.


--Josh Berkus

Please note new address & phone number below to be in effect as of Nov 25th.

Please update your records.

Jeff Rule | Distinguished Engineer
DemandTec, Inc. | 1 Franklin Parkway, Building 910, Suite 200 | San Mateo, CA 94403
t | 650.645.7110 f | 650.645.7377 m | 415.706.4928

www.demandtec.com

******************************************************************
DemandTec Email Notice

This email and any attachments may contain confidential and/or proprietary
information and is intended solely for the use of the addressee. If you are not
the intended recipient we request that you notify us via email or telephone and
delete all copies of the message from your systems. Additionally, although
DemandTec has taken reasonable precautions to ensure the security of this
email and any attachments, we encourage you to take similar precautions and
accept no liability for any loss or damage resulting from its use.

DemandTec, 1 Franklin Parkway, Building 910, San Mateo, CA 94403-1906, 650-650-7100

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:31:06Z

Cool, Thanks for the detailed response. I've certainly not done a ton of research comparing different VPS solutions myself, but I was under the (quite possible mis-impression) that at EC2 the underlying hardware was not 1:1 related to your performance (eg you get a set amount of cpu throughput, and if they had older hardware underneath, then you'd just get more of it). I also did not realize other users could steal cycles from you like happens on most other VPS offerings. I haven't seen that much documentation to base any of these assumptions on, of course, so it's good to hear your perspective.

Some posts, such as http://journal.uggedal.com/vps-performance-comparison, seem to imply different conclusions (not that he's suggesting EC2 is a good deal, but for different reasons than cpu stealing), and I'd love to see similar posts on the topic. I'd be happy to switch to something else if I felt I was going to getting a much better deal (I really do like the integrated EBS/S3/Cloudfront options tho for the type of projects I generally work on).

Interesting and useful

Cheers,

-Josh

On Thu, Dec 10, 2009 at 11:20 AM, Josh Berkus <josh@...> wrote:

Josh,

> Pure curiosity on my part here ... I use EC2 a bit, tho not as much as
> the serious users. A few large and small instances on all the time, and
> I boot up new ones for shorter periods all the time. First - I've
> never had any issue getting my instances fulfilled right away (I always
> use EAST-C, but perhaps other datacenters are generally more full, or
> you are trying to boot up many tens of servers at once?).

Yeah, the two issues I've had are (a) requisitioning high-end instances
(like 32G/16core instances) and (b) allocating a lot at once. Sometimes
instances just "aren't available" and there's no way to find out when
they will be available.

> Also, when you say they are slow, do you mean in terms of $/cycle,
> or you wish you had burst access to other users unused cycles like on
> some other vps offerings? something else?

I mean that if you have an 8core/16GB instance, the actual processing
throughput you get is about 1/6 to 1/4 that of a new HP DL380 machine
with 8cores and 16GB. So you really need 4x as many EC2 instances to
match bare metal. Partly this is due to CPU-stealing, and partly to
erratic and lag-prone I/O, and partly to the fact that a lot of machines
in the EC2 pool are 4 years old.

> I like the bundle of offerings that AWS provides (EBS, especially), and
> I've personally had great experience w/them (fwiw I've also had great
> experience w/slicehost) -- but if I am getting missing out on how
> they're screwing me, for example by stealing my CPU, I'd definitely love
> to learn more.

On EC2, other VMs on the same hardware are permitted to "steal" portions
of the CPU which are allocated to you. So at any given time, you may
have as little as 50% of the CPUs you're being billed for. And, when
CPU availability is fluctuating up and down (as it does on EC2), real
throughput tends to be based on the slowest second rather than peak
availablity. Most Linux apps, especially databases, do quite poorly
with erratic resource availability.

--Josh

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:30:37Z

Some posts, such as http://journal.uggedal.com/vps-performance-comparison, seem to imply different conclusions (not that he's suggesting EC2 is a good deal, but for different reasons than cpu stealing), and I'd love to see similar posts on the topic: I'd be happy to switch to something else if I felt I was going to getting a much better deal (I really do like the integrated EBS/S3/Cloudfront options tho for the type of projects I generally work on).

Cheers,

-Josh

On Thu, Dec 10, 2009 at 11:20 AM, Josh Berkus <josh@...> wrote:

Josh,

> Pure curiosity on my part here ... I use EC2 a bit, tho not as much as
> the serious users. A few large and small instances on all the time, and
> I boot up new ones for shorter periods all the time. First - I've
> never had any issue getting my instances fulfilled right away (I always
> use EAST-C, but perhaps other datacenters are generally more full, or
> you are trying to boot up many tens of servers at once?).

Yeah, the two issues I've had are (a) requisitioning high-end instances
(like 32G/16core instances) and (b) allocating a lot at once. Sometimes
instances just "aren't available" and there's no way to find out when
they will be available.

> Also, when you say they are slow, do you mean in terms of $/cycle,
> or you wish you had burst access to other users unused cycles like on
> some other vps offerings? something else?

I mean that if you have an 8core/16GB instance, the actual processing
throughput you get is about 1/6 to 1/4 that of a new HP DL380 machine
with 8cores and 16GB. So you really need 4x as many EC2 instances to
match bare metal. Partly this is due to CPU-stealing, and partly to
erratic and lag-prone I/O, and partly to the fact that a lot of machines
in the EC2 pool are 4 years old.

> I like the bundle of offerings that AWS provides (EBS, especially), and
> I've personally had great experience w/them (fwiw I've also had great
> experience w/slicehost) -- but if I am getting missing out on how
> they're screwing me, for example by stealing my CPU, I'd definitely love
> to learn more.

On EC2, other VMs on the same hardware are permitted to "steal" portions
of the CPU which are allocated to you. So at any given time, you may
have as little as 50% of the CPUs you're being billed for. And, when
CPU availability is fluctuating up and down (as it does on EC2), real
throughput tends to be based on the slowest second rather than peak
availablity. Most Linux apps, especially databases, do quite poorly
with erratic resource availability.

--Josh

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:20:23Z

Josh,

> Pure curiosity on my part here ... I use EC2 a bit, tho not as much as
> the serious users. A few large and small instances on all the time, and
> I boot up new ones for shorter periods all the time. First - I've
> never had any issue getting my instances fulfilled right away (I always
> use EAST-C, but perhaps other datacenters are generally more full, or
> you are trying to boot up many tens of servers at once?).

Yeah, the two issues I've had are (a) requisitioning high-end instances
(like 32G/16core instances) and (b) allocating a lot at once. Sometimes
instances just "aren't available" and there's no way to find out when
they will be available.

> Also, when you say they are slow, do you mean in terms of $/cycle,
> or you wish you had burst access to other users unused cycles like on
> some other vps offerings? something else?

I mean that if you have an 8core/16GB instance, the actual processing
throughput you get is about 1/6 to 1/4 that of a new HP DL380 machine
with 8cores and 16GB. So you really need 4x as many EC2 instances to
match bare metal. Partly this is due to CPU-stealing, and partly to
erratic and lag-prone I/O, and partly to the fact that a lot of machines
in the EC2 pool are 4 years old.

> I like the bundle of offerings that AWS provides (EBS, especially), and
> I've personally had great experience w/them (fwiw I've also had great
> experience w/slicehost) -- but if I am getting missing out on how
> they're screwing me, for example by stealing my CPU, I'd definitely love
> to learn more.

On EC2, other VMs on the same hardware are permitted to "steal" portions
of the CPU which are allocated to you. So at any given time, you may
have as little as 50% of the CPUs you're being billed for. And, when
CPU availability is fluctuating up and down (as it does on EC2), real
throughput tends to be based on the slowest second rather than peak
availablity. Most Linux apps, especially databases, do quite poorly
with erratic resource availability.

--Josh

Re: Recommended/Not Recommended Hosts?

2009-12-10T11:02:12Z

-- Amazon EC2: uptime and availability are great, but the servers are
sloooooooow and fulfillment of new instances is unreliable. Also,
CPU-stealing.

Pure curiosity on my part here ... I use EC2 a bit, tho not as much as the serious users. A few large and small instances on all the time, and I boot up new ones for shorter periods all the time. First - I've never had any issue getting my instances fulfilled right away (I always use EAST-C, but perhaps other datacenters are generally more full, or you are trying to boot up many tens of servers at once?).

Also, when you say they are slow, do you mean in terms of $/cycle, or you wish you had burst access to other users unused cycles like on some other vps offerings? something else?

I like the bundle of offerings that AWS provides (EBS, especially), and I've personally had great experience w/them (fwiw I've also had great experience w/slicehost) -- but if I am getting missing out on how they're screwing me, for example by stealing my CPU, I'd definitely love to learn more.

Re: Recommended/Not Recommended Hosts?

2009-12-10T10:52:25Z

On Thu, Dec 10, 2009 at 10:43 AM, Josh Berkus <josh@...> wrote:
> What hosts, both virtual hosts and colos, do you recommend for
> PostgreSQL-based applications, and which ones have you had bad
> experiences with? If we get a list together, we'll put it up somewhere
> community.
>
> For my part, I've had reasonably good experiences with:
>
> -- Layer42 for low-end colo, but watch your bandwith, they won't.

At the risk of giving away one of my best kept secrets, Layer42 is
simply the most outstanding and professional colocation provider I
have ever had the pleasure to work with. Their network is FAST and
unbelievably reliable. Steve and his team are of the highest caliber.
After going through half a dozen different providers of virtual hosts
and colocations, I finally found a provider that I can trust with my
mission critical needs.

They are located in Santa Clara which is a bit of a drive, but
considering I only need to ever make the trip to add more capacity, I
don't mind it one bit. I highly recommend them.

Recommended/Not Recommended Hosts?

2009-12-10T10:43:47Z

All,

What hosts, both virtual hosts and colos, do you recommend for
PostgreSQL-based applications, and which ones have you had bad
experiences with? If we get a list together, we'll put it up somewhere
community.

For my part, I've had reasonably good experiences with:

-- Rackspace rental servers (provided you're OK with Dell hardware)
-- Layer42 for low-end colo, but watch your bandwith, they won't.
-- Joyent for online data warehousing (if you can deal with Solaris)

I've had bad experiences with:

-- GoGrid Cloud Servers ... I/O is very very bad, so your database
better fit in RAM, and they're limited to 8GB max. On the other hand,
no downtime whatsoever. (in GG's defense, they recommend colo for databases)

-- Amazon EC2: uptime and availability are great, but the servers are
sloooooooow and fulfillment of new instances is unreliable. Also,
CPU-stealing.

--Josh Berkus

[Fwd: She's Geeky invite for women]

2009-12-10T10:01:59Z

-------- Original Message --------
Subject: [Foundations] She's Geeky invite for women
Date: Thu, 10 Dec 2009 01:06:28 -0800
From: kaliya@...
To: foundations@...

So one of the issues that has been circulating in the tech world for
some time has been the issue of women (and other diversity) along with
the challenge of how to address it.

2 years ago I founded a women's technology unconference to support women
sharing and learning from one another with the ultimate goal of helping
those women who are geeky feeling more confident in their geekiness to
actually come out and express themselves more often in mixed settings.

We are having our 3rd Bay Area event at the end of January and I wanted
to extend an invitation to all the women on this foundations list to
attend. Unfortunately this event is for women only - we have thought a
lot about this choice and why we made it - it is decidedly not about be
being "anti-male" and mostly about creating a safe space for women who
have been shy to express themselves. If this makes you feel
uncomfortable you can read more about this nuance and what we are about
here. http://www.shesgeeky.org/about <ttp://www.shesgeeky.org/about">

Please consider passing this along to your over all FLOSS communities or
just women you know who participate - knowing about it and attending it
could make a big difference for women in your communities.

Thanks,
-Kaliya

*She’s Geeky: Connecting Women in Tech*
Returning to the Bay Area January 29, 30 & 31, 2010
@ the Computer History Museum in Mountain View

*/She's Geeky is just 7 weeks away! /*
*/Early Bird Tickets are available for just 2 more weeks until December
20th. /*

http://shesgeekybayarea3.eventbrite.com/
<ttp://shesgeekybayarea3.eventbrite.com/">
www.shesgeeky. <ttp://www.shesgeeky.com/">org

*This event is for:***

* Women Working in Science, Technology, Engineering and Mathematic
Fields.
* Women into their gadgets and SciFi Fan’s.
* Women students of the sciences, those training to be engineers,
aspiring mathematicians and technology professions.
* Women who are kernel hackers and all those who aspire to deepen
their geekiness by learning how to code the php on their blogs.
* Daughters, Nieces and mentees of all of the above
* Women who are retired used to work in tech related fields.

*What happens?***

* All the women who attend are invited to create the agenda live the
day of the event.
* Women can present/share about their area of professional expertise.
* Women learn from one another.
* Women discuss critical issues affecting them in the digital age.
* Women talk about work place and community issues they face.
* Women are inspired to follow their passion and believe in their
own abilities.
* Women find connections and support for their work and vision.

*About The Format*
She’s Geeky is an unConference
<ttp://www.unconference.net/"> (http://www.unconference.net/
<ttp://www.unconference.net/">) where the agenda is created by all
participants live the day the event happens. This format supports peer
to peer learning, dialogue about the issues that are top of mind and
networking. In this women’s only environment attendees have the
opportunity to see their contribution to their field in a new light and
gain confidence to step forward in their lives and careers.

Click here
<ttp://shesgeeky.org/sg/2009/11/twitter-highlights-from-shes-geeky-dc/">
(http://shesgeeky.org/sg/2009/11/twitter-highlights-from-shes-geeky-dc/
<ttp://shesgeeky.org/sg/2009/11/twitter-highlights-from-shes-geeky-dc/">) for
a dip into the Twitter Stream from the November 13 & 14, 2009, sold out,
Washington, DC She’s Geeky Event to get a sense of the experience from
those who attended! Or to read answers to the end of day question: ‘As
a result of today…’ click here <ttp://bit.ly/4pXVwI">.
(http://www.shesgeeky.org/wiki/Sg2009dc:Results
<ttp://www.shesgeeky.org/wiki/Sg2009dc:Results">)

*About She's Geeky*
She’s Geeky convenes to inspire women for the future, providing a
gathering space to create enduring communities that foster collaboration
and innovation, while promoting initiative and leadership among women
tech professionals. Beginning with its resoundingly successful 2007
unConference in Silicon Valley, She’s Geeky attracts women from a broad
spectrum of technological specialties, diverse social groups,
generations, and levels of expertise. The inclusive quality of She’s
Geeky events promotes discussion, furthers cooperation, and encourages
learning. She’s Geeky advances systemic change in tech culture by
disseminating effective practices to address the challenges of women
working in Science, Technology, Engineering and Mathematics.

She’s Geeky isn’t a “women in tech group” and we don’t have chapters
around the country. We are an unConference event that works to connect
and promote existing tech groups. Currently we work with DevChix,
LinuxChix, Women 2.0, Women Who Tech, Digital Sistas, Girl Geek Dinners,
Gaming Angels, the Anita Borg Institute for Women in Technology.

*We hope you will join us! ***
*
*
If you want to post about the event you can find graphics here along
with a PDF flyer.
http://shesgeeky.org/sg/graphics/ <ttp://shesgeeky.org/sg/graphics/">

If you have questions we can be reached via info@...
<ailto:info@...">

You can find us on Twitter at - http://www.twitter.com/shesgeeky
<ttp://www.twitter.com/shesgeeky">
Our website is http://www.shesgeeky.org <ttp://www.shesgeeky.org/">

_______________________________________________
foundations mailing list
foundations@...
http://lists.freedesktop.org/mailman/listinfo/foundations

Re: [GENERAL] Tuesday (PST8PDT) Jeff Davis Presents: Operator Exclusion Constraints

2009-12-08T10:10:50Z

> I suppose the video will be available later online?

Yes.

--Josh

Re: [GENERAL] Tuesday (PST8PDT) Jeff Davis Presents: Operator Exclusion Constraints

2009-12-08T10:10:30Z

On Tue, Dec 08, 2009 at 09:02:41AM +0100, Guillaume Lelarge wrote:

> Hi David,
>
> Le mardi 8 décembre 2009 à 01:46:00, David Fetter a écrit :
> > Folks,
> >
> > If you can't make it to the SFPUG meeting in person on Tuesday,
> > December 8, 2009 at EZRez
> > http://postgresql.meetup.com/1/calendar/11928447/
> >
> > You can see the live stream at:
> > http://media.postgresql.org/sfpug/streaming/
> >
> > We'll also be using an irc channel for questions, comments, etc:
> > irc://irc.freenode.net/sfpug
> >
>
> I suppose the video will be available later online?

Yes.

I'll send out notices when it's up :)

Cheers,
David.
--
David Fetter <david@...> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@...
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: [GENERAL] Tuesday (PST8PDT) Jeff Davis Presents: Operator Exclusion Constraints

2009-12-08T00:02:41Z

Hi David,

Le mardi 8 décembre 2009 à 01:46:00, David Fetter a écrit :

> Folks,
>
> If you can't make it to the SFPUG meeting in person on Tuesday,
> December 8, 2009 at EZRez
> http://postgresql.meetup.com/1/calendar/11928447/
>
> You can see the live stream at:
> http://media.postgresql.org/sfpug/streaming/
>
> We'll also be using an irc channel for questions, comments, etc:
> irc://irc.freenode.net/sfpug
>

I suppose the video will be available later online?

Cheers.

--
Guillaume.
http://www.postgresqlfr.org
http://dalibo.com

Tuesday (PST8PDT) Jeff Davis Presents: Operator Exclusion Constraints

2009-12-07T16:46:00Z

Folks,

If you can't make it to the SFPUG meeting in person on Tuesday,
December 8, 2009 at EZRez
http://postgresql.meetup.com/1/calendar/11928447/

You can see the live stream at:
http://media.postgresql.org/sfpug/streaming/

We'll also be using an irc channel for questions, comments, etc:
irc://irc.freenode.net/sfpug

Cheers,
David.
--
David Fetter <david@...> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@...
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Location set for Tuesday SFPUG, but bring $ for food

2009-12-03T16:33:13Z

All,

EzRez has generously offered to host our meetup again. However, they
can't sponsor food, so if you plan to eat, bring $10 IN CASH. We'll
produce Indian, Thai, Mexican or Chinese food (including vegetarian) to
share.

EzRez San Francisco
50 1st Street 7th Floor
San Francisco, CA 94105

http://postgresql.meetup.com/1/calendar/11928447/
When: December 8, 2009 7:00 PM

December SFPUG: Exclusion constraints, and need host!

2009-11-21T14:55:32Z

All,

We've scheduled a talk from Jeff Davis on his current work on 8.5, which
you'll understand is really cool if you've ever built a scheduling
application. However, we need a meeting host in the city of San
Francisco, preferably one who can sponsor food.

What: Operator Exclusion Constraints
Who: Jeff Davis
Where: in San Francisco, exact location TBA
When: Tuesday, Dec. 8th, 7pm to 9pm

Jeff Davis will present his new feature for PostgreSQL 8.5 which will
(hopefully) be committed by the time of the meeting. While the name of
the feature is obscure, its uses are compelling for anyone who has ever
had to build a calendaring application, used PostGIS, or otherwise needs
to define a complex concept of uniqueness to their database design. With
Exclusion Constraints, you can make anything unique which you can define
as an operator.

http://postgresql.meetup.com/1/calendar/1192844

ANyone SOMA want a ride to SFPUG tonight?

2009-11-10T11:58:54Z

All,

Is there anyone south of market in SF who wants a ride to SFPUG tonight?
We'd like to use the carpool lane ...

--Josh Berkus