Is Replicating Data from Firewalled Master Possible?

[spencer.c]

I need to replicate data from a server inside our firewall to a server outside our firewall.  The server outside of our firewall will not have any access through the firewall, so the typical MySQL replication setup doesn't seem viable, as to use it the slave has to be able to connect to the master to retrieve the data.

Is there a straightforward way to replicate the data from the internal server to the outside server in this scenario?  I just want to push the data out to a single receiver, on an ongoing basis, rather than have the external server pull it down.

Thanks for any help.

[Augusto Bott-4]

Why don't you just copy the binlogs from time to time from the master to the
slave and then apply them in batches?
This way you don't need the slave to connect to its master (since the
binlogs would be applied manually or via a script one by one)

--
Augusto Bott

On 7/4/07, spencer.c <spencer.crissman@...> wrote:

>
>
> I need to replicate data from a server inside our firewall to a server
> outside our firewall.  The server outside of our firewall will not have
> any
> access through the firewall, so the typical MySQL replication setup
> doesn't
> seem viable, as to use it the slave has to be able to connect to the
> master
> to retrieve the data.
>
> Is there a straightforward way to replicate the data from the internal
> server to the outside server in this scenario?  I just want to push the
> data
> out to a single receiver, on an ongoing basis, rather than have the
> external
> server pull it down.
>
> Thanks for any help.
>
>
> --
> View this message in context:
> http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Master-Possible--tf4026508.html#a11437456
> Sent from the MySQL - Replication mailing list archive at Nabble.com.
>
>
> --
> MySQL Replication Mailing List
> For list archives: http://lists.mysql.com/replication
> To unsubscribe:
> http://lists.mysql.com/replication?unsub=augusto.bott@...
>
>

[spencer.c]

Thanks for the feedback.  It looks like that will work.  I will make a script on the master that runs the logs through mysqlbinlog and pipes the commands to the external server.  

When I am doing this, if I specify -d dbname for mysqlbinlog, will I need to have selected the appropriate database in my mysql connection before sending the commands?  Browsing through the output from the mysqlbinlog, I don't see any "USE DATABASE" commands or anything that would direct the queries to specific databases, so I was just curious about that.

Anyway, thanks again for getting me pointed in the right direction.

Augusto Bott-4 wrote:

Why don't you just copy the binlogs from time to time from the master to the
slave and then apply them in batches?
This way you don't need the slave to connect to its master (since the
binlogs would be applied manually or via a script one by one)

--
Augusto Bott

On 7/4/07, spencer.c <[e-mail]> wrote:
>
>
> I need to replicate data from a server inside our firewall to a server
> outside our firewall.  The server outside of our firewall will not have
> any
> access through the firewall, so the typical MySQL replication setup
> doesn't
> seem viable, as to use it the slave has to be able to connect to the
> master
> to retrieve the data.
>
> Is there a straightforward way to replicate the data from the internal
> server to the outside server in this scenario?  I just want to push the
> data
> out to a single receiver, on an ongoing basis, rather than have the
> external
> server pull it down.
>
> Thanks for any help.
>
>
> --
> View this message in context:
> http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Master-Possible--tf4026508.html#a11437456
> Sent from the MySQL - Replication mailing list archive at Nabble.com.
>
>
> --
> MySQL Replication Mailing List
> For list archives: http://lists.mysql.com/replication
> To unsubscribe:
> http://lists.mysql.com/replication?unsub=[e-mail]
>
>

[Augusto Bott-4]

I'd recommend you to check
http://dev.mysql.com/doc/refman/5.0/en/replication-rules.html for how this
'USE database' thing actually works with the binlog. But in a short answer:
yes, if you are extracting from the binlog the statements from only one
database, you should specify it when loading to the destination server.

--
Augusto Bott

On 7/5/07, spencer.c <spencer.crissman@...> wrote:

>
>
> Thanks for the feedback.  It looks like that will work.  I will make a
> script
> on the master that runs the logs through mysqlbinlog and pipes the
> commands
> to the external server.
>
> When I am doing this, if I specify -d dbname for mysqlbinlog, will I need
> to
> have selected the appropriate database in my mysql connection before
> sending
> the commands?  Browsing through the output from the mysqlbinlog, I don't
> see
> any "USE DATABASE" commands or anything that would direct the queries to
> specific databases, so I was just curious about that.
>
> Anyway, thanks again for getting me pointed in the right direction.
>
>
>
>
>
> Augusto Bott-4 wrote:
> >
> > Why don't you just copy the binlogs from time to time from the master to
> > the
> > slave and then apply them in batches?
> > This way you don't need the slave to connect to its master (since the
> > binlogs would be applied manually or via a script one by one)
> >
> > --
> > Augusto Bott
> >
> > On 7/4/07, spencer.c <spencer.crissman@...> wrote:
> >>
> >>
> >> I need to replicate data from a server inside our firewall to a server
> >> outside our firewall.  The server outside of our firewall will not have
> >> any
> >> access through the firewall, so the typical MySQL replication setup
> >> doesn't
> >> seem viable, as to use it the slave has to be able to connect to the
> >> master
> >> to retrieve the data.
> >>
> >> Is there a straightforward way to replicate the data from the internal
> >> server to the outside server in this scenario?  I just want to push the
> >> data
> >> out to a single receiver, on an ongoing basis, rather than have the
> >> external
> >> server pull it down.
> >>
> >> Thanks for any help.
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Master-Possible--tf4026508.html#a11437456
> >> Sent from the MySQL - Replication mailing list archive at Nabble.com.
> >>
> >>
> >> --
> >> MySQL Replication Mailing List
> >> For list archives: http://lists.mysql.com/replication
> >> To unsubscribe:
> >> http://lists.mysql.com/replication?unsub=augusto.bott@...
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Master-Possible--tf4026508.html#a11452754
> Sent from the MySQL - Replication mailing list archive at Nabble.com.
>
>
> --
> MySQL Replication Mailing List
> For list archives: http://lists.mysql.com/replication
> To unsubscribe:
> http://lists.mysql.com/replication?unsub=augusto.bott@...
>
>

[Frazer Clement-2]

You might be able to do something with SSH port forwarding -
http://www.ssh.com/support/documentation/online/ssh/adminguide/32/Port_Forwarding.html
The internal server can SSH out through the firewall to the external
server, and set up a port forward back to itself.
Reliability might be an issue - if the tunnel fails you need to detect it
and recover it somehow





"spencer.c" <spencer.crissman@...>
04/07/2007 22:01

To
replication@...
cc

Subject
Is Replicating Data from Firewalled Master Possible?







I need to replicate data from a server inside our firewall to a server
outside our firewall.  The server outside of our firewall will not have
any
access through the firewall, so the typical MySQL replication setup
doesn't
seem viable, as to use it the slave has to be able to connect to the
master
to retrieve the data.

Is there a straightforward way to replicate the data from the internal
server to the outside server in this scenario?  I just want to push the
data
out to a single receiver, on an ongoing basis, rather than have the
external
server pull it down.

Thanks for any help.


--
View this message in context:
http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Master-Possible--tf4026508.html#a11437456

Sent from the MySQL - Replication mailing list archive at Nabble.com.


--
MySQL Replication Mailing List
For list archives: http://lists.mysql.com/replication
To unsubscribe:    
http://lists.mysql.com/replication?unsub=f.clement@...








Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Rick James]

FLUSH LOGS will make this approach simpler -- then you are copying whole
files, and the position is always start-of-file.

Port-forwarding mentioned later is probably better.

> -----Original Message-----
> From: augusto.bott@... [mailto:augusto.bott@...]
> On Behalf Of Augusto Bott
> Sent: Wednesday, July 04, 2007 3:24 PM
> To: spencer.c
> Cc: replication@...
> Subject: Re: Is Replicating Data from Firewalled Master Possible?
>
> Why don't you just copy the binlogs from time to time from
> the master to the
> slave and then apply them in batches?
> This way you don't need the slave to connect to its master (since the
> binlogs would be applied manually or via a script one by one)
>
> --
> Augusto Bott
>
> On 7/4/07, spencer.c <spencer.crissman@...> wrote:
> >
> >
> > I need to replicate data from a server inside our firewall
> to a server
> > outside our firewall.  The server outside of our firewall
> will not have
> > any
> > access through the firewall, so the typical MySQL replication setup
> > doesn't
> > seem viable, as to use it the slave has to be able to connect to the
> > master
> > to retrieve the data.
> >
> > Is there a straightforward way to replicate the data from
> the internal
> > server to the outside server in this scenario?  I just want
> to push the
> > data
> > out to a single receiver, on an ongoing basis, rather than have the
> > external
> > server pull it down.
> >
> > Thanks for any help.
> >
> >
> > --
> > View this message in context:
> >
> http://www.nabble.com/Is-Replicating-Data-from-Firewalled-Mast
> er-Possible--tf4026508.html#a11437456
> > Sent from the MySQL - Replication mailing list archive at
> Nabble.com.
> >
> >
> > --
> > MySQL Replication Mailing List
> > For list archives: http://lists.mysql.com/replication
> > To unsubscribe:
> > http://lists.mysql.com/replication?unsub=augusto.bott@...
> >
> >
>

--
MySQL Replication Mailing List
For list archives: http://lists.mysql.com/replication
To unsubscribe:    http://lists.mysql.com/replication?unsub=lists@...