Sequential Userid/Password reads

[Mark Davis-3]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Setup: 4 Agent Machines, running 4 processes (JVMs), with 20 threads each, for 5 runs.
This will send a total of 1600 users through my scenario.

I need each individual thread to get a unique userid from a file.  The file contains 2000 entries, so random functions will not work, as we need to create updates with unique userids.
How would I manage multiple threads reading the same file?
Separate userid files on each box is where I started.  But, I really need a way for all the threads on each box to coordinate the reading of the file.

Has anyone found a successful way to do this?

TIA
Mark

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Travis Bear]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Mark,

I wrote an Apache log playback Grinder script a few weeks back and faced the same question of how to properly have many threads reading a single file.  My approach was to have a single thread read the file, dumping each line into a python queue.  All the worker threads then pulled the data they needed from the queue, rather than directly form the file.

Here's the implementation I went with:
http://track.svn.sourceforge.net/viewvc/track/grinderScripts/filePlayback.py?revision=266&view=markup

Hope that's helpful.


-Travis

---

From: Mark Davis <markdavis65@...>
To: grinder-use@...
Sent: Thu, October 29, 2009 2:29:21 PM
Subject: [Grinder-use] Sequential Userid/Password reads

Setup: 4 Agent Machines, running 4 processes (JVMs), with 20 threads each, for 5 runs.
This will send a total of 1600 users through my scenario.

I need each individual thread to get a unique userid from a file.  The file contains 2000 entries, so random functions will not work, as we need to create updates with unique userids.
How would I manage multiple threads reading the same file?
Separate userid files on each box is where I started.  But, I really need a way for all the threads on each box to coordinate the reading of the file.

Has anyone found a successful way to do this?

TIA
Mark

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Anirvan]

Hello Mark,

I had managed to achieve this by implementing a logic which was something like -
1. Read all the user IDs and passwords, and create an  array.
2. In your Grinder test script when you invoke the method you've written to retrieve a particular user ID & pwd from the array of step 1, pass the thread number as an argument.
3. Return the user ID and pwd for the particular thread number.

Grinder ensures that each thread has a unique identifier number. So you'll never have 2 threads having the same index number. However, in adopting this approach you need to be sure that the number of user IDs is >= the number of threads you define for testing your application.

Hope this helps!
anirvan

On Fri, Oct 30, 2009 at 2:59 AM, Mark Davis <markdavis65@...> wrote:

Setup: 4 Agent Machines, running 4 processes (JVMs), with 20 threads each, for 5 runs.
This will send a total of 1600 users through my scenario.

I need each individual thread to get a unique userid from a file.  The file contains 2000 entries, so random functions will not work, as we need to create updates with unique userids.
How would I manage multiple threads reading the same file?
Separate userid files on each box is where I started.  But, I really need a way for all the threads on each box to coordinate the reading of the file.

Has anyone found a successful way to do this?

TIA
Mark

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[kamalakar vadla]

Hi Anirvan,

Your approach works fine in the case single agent but, Mark wanted is something which works with multiple agents rt...?
 your approach will fail in the case of multiple agents

Case 1:
Maintain g different prop file for different agents then your approach works fine .

Case 2:
If user want to maintain single file from where all agents should pick the credentials then this will fail because each agent starts thread number zero.

Hi Travis,

I was looking at ur code its nice but i am was unable to understand how actually it works.. Can you provide me how its works and if possible any documentation about this helps a lot


Thanks in advance
Kamalakar

On Fri, Oct 30, 2009 at 11:18 AM, Anirban Majumdar <anirvan.majumdar@...> wrote:

Hello Mark,

I had managed to achieve this by implementing a logic which was something like -
1. Read all the user IDs and passwords, and create an  array.
2. In your Grinder test script when you invoke the method you've written to retrieve a particular user ID & pwd from the array of step 1, pass the thread number as an argument.
3. Return the user ID and pwd for the particular thread number.

Grinder ensures that each thread has a unique identifier number. So you'll never have 2 threads having the same index number. However, in adopting this approach you need to be sure that the number of user IDs is >= the number of threads you define for testing your application.

Hope this helps!
anirvan

On Fri, Oct 30, 2009 at 2:59 AM, Mark Davis <markdavis65@...> wrote:

Setup: 4 Agent Machines, running 4 processes (JVMs), with 20 threads each, for 5 runs.
This will send a total of 1600 users through my scenario.

I need each individual thread to get a unique userid from a file.  The file contains 2000 entries, so random functions will not work, as we need to create updates with unique userids.
How would I manage multiple threads reading the same file?
Separate userid files on each box is where I started.  But, I really need a way for all the threads on each box to coordinate the reading of the file.

Has anyone found a successful way to do this?

TIA
Mark

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list

grinder-use@...

https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

--
Kamalakar.Vadla
Pramati Technologies Pvt Ltd
Begumpet

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Jörg Kalsbach]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Vadla,

 

If you look at http://www.jk-itberatung.de/grinderinthecloud.html chapter “Distribute userdata over agents” you’ll find example code for a working approach. It’ basically the same as Anirban idea.

 

Cheers,

 

Jörg

 

---

IT-Beratung Jörg Kalsbach

Ubierring 25

D-50678 Köln

 

Tel :      [phone]

Email :    j.kalsbach@...

 

USt-IdNr.: DE258604074

 

 

Von: kamalakar vadla [mailto:vkchary417@...]
Gesendet: Friday, October 30, 2009 8:19 AM
An: grinder-use
Betreff: Re: [Grinder-use] Sequential Userid/Password reads

 

Hi Anirvan,

Your approach works fine in the case single agent but, Mark wanted is something which works with multiple agents rt...?
 your approach will fail in the case of multiple agents

Case 1:
Maintain g different prop file for different agents then your approach works fine .

Case 2:
If user want to maintain single file from where all agents should pick the credentials then this will fail because each agent starts thread number zero.

Hi Travis,

I was looking at ur code its nice but i am was unable to understand how actually it works.. Can you provide me how its works and if possible any documentation about this helps a lot


Thanks in advance
Kamalakar

On Fri, Oct 30, 2009 at 11:18 AM, Anirban Majumdar <anirvan.majumdar@...> wrote:

Hello Mark,

I had managed to achieve this by implementing a logic which was something like -
1. Read all the user IDs and passwords, and create an  array.
2. In your Grinder test script when you invoke the method you've written to retrieve a particular user ID & pwd from the array of step 1, pass the thread number as an argument.
3. Return the user ID and pwd for the particular thread number.

Grinder ensures that each thread has a unique identifier number. So you'll never have 2 threads having the same index number. However, in adopting this approach you need to be sure that the number of user IDs is >= the number of threads you define for testing your application.

Hope this helps!
anirvan

On Fri, Oct 30, 2009 at 2:59 AM, Mark Davis <markdavis65@...> wrote:

Setup: 4 Agent Machines, running 4 processes (JVMs), with 20 threads each, for 5 runs.
This will send a total of 1600 users through my scenario.

I need each individual thread to get a unique userid from a file.  The file contains 2000 entries, so random functions will not work, as we need to create updates with unique userids.
How would I manage multiple threads reading the same file?
Separate userid files on each box is where I started.  But, I really need a way for all the threads on each box to coordinate the reading of the file.

Has anyone found a successful way to do this?

TIA
Mark

 

 

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list

grinder-use@...

https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

--
Kamalakar.Vadla
Pramati Technologies Pvt Ltd
Begumpet

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[nick3216]

> I need each individual thread to get a unique userid from a file.
> How would I manage multiple threads reading the same file?

The method below is how I do this, and it also extrapolates to multiple
processes, and could be expanded for agents. In my case I have one file
per agent because I needed 6000 users, and rather more information per
user than shown below, and consequently hit the Jython character limit.

Create a file of testusers for each agent, named LoadTestUsers{n}.py
where {n} is a number from 0 to number-of-agents - 1. Each file taskes
the form:

  numUsers = 1000 # or however many users you have in the file
  usersAgent0 = (
    ('[login-name]', '[password]'),
    ...
    )

In the test script each agent chooses which LoadTestUsers file to
import:

  Agent=str(grinder.getAgentNumber())
  userModule = 'LoadTestUsers' + agent
  userImportCommand = "from " + userModule + " import usersAgent" +
agent " as users"
  Exec userImportCommand
  numUsersImportCommand = "from " + userModule + " import numUsers as
numUsers"
  Exec userImportCommand

That gets around each agent having access to different users. Now to
handle processes and threads...

In the thread class constructor users are divided up amongst the actual
number of processes and threads running this test, so each thread works
out a base index in the users file where it should start reading users:

  def __init__(self):
        processes = properties.getInt("grinder.processes", 10)
        threads = properties.getInt("grinder.threads"), 10)
        usersPerProcess = numUsers / processes
        self.usersPerThread = usersPerProcess /
properties.getInt(grinder.threads), 10)
        self.baseIndex = ((grinder.getProcessNumber % processes) *
usersPerProcess)) + (grinder.GetThreadNumber() * self.usersPerThread))

Then each run uses this baseindex to determine which user to load on
each run...

  def __call__(self):
    run = grinder.getRunNumber()
    if (run >= self.usersPerThread):
      run = 0
    index = self.baseIndex + run
    login = users[index][0]
    password = users[index][1]

    # If you need convincing that each thread across all your agents,
    # processes and threads, is choosing different, repeatable, users
    # then log the users used by each thread and run in a form
    # convenient to you.
    # I use Test Anything Protocol
[http://en.wikipedia.org/wiki/Test_Anything_Protocol]
    DumpDataToTAP(
      {'agent':grinder.getAgentNumber(),
       'process':grinder.getProcessNumber(),
       'thread':grinder.getThreadNumber(),
       'run':run,
       'userindex':index,
       'login':login})

    ...
   

    # Then in the test itself page 1 is parameterised to take login and
password
    self.page1(login, password)


It may not be the prettiets or cleverest method but it works.

Nick Wallis

This e-mail from Ultra Electronics Limited and any attachments to it are confidential to the intended recipient and may also be privileged.  If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person.  All communications may be subject to interception or monitoring for operational and/or security purposes.  Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. Ultra Electronics Limited is a company registered in England and Wales, registration number 2830644. The address of its registered office is 417 Bridport Road, Greenford, Middlesex, UB6 8UA.


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use