[PATCH 3/8] SGI x86_64 UV: Limit the number of number of SRAT messages

[Mike Travis]

Limit number of SRAT messages of the form:

[    0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0

Cc: H. Peter Anvin <hpa@...>
Cc: x86@...
Cc: David Rientjes <rientjes@...>
Cc: Yinghai Lu <yinghai@...>
Cc: Mel Gorman <mel@...>
Cc: linux-kernel@...
Signed-off-by: Mike Travis <travis@...>
---
 arch/x86/mm/srat_64.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

--- linux.orig/arch/x86/mm/srat_64.c
+++ linux/arch/x86/mm/srat_64.c
@@ -136,8 +136,9 @@
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+ if (node < 2 || !limit_console_output(false))
+ printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
+       pxm, apic_id, node);
 }
 
 /* Callback for Proximity Domain -> LAPIC mapping */
@@ -170,8 +171,9 @@
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+ if (node < 2 || !limit_console_output(false))
+ printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
+       pxm, apic_id, node);
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE

--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Andi Kleen-3]

Mike Travis <travis@...> writes:

> Limit number of SRAT messages of the form:
> [    0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0

While I generally agree on the concept of limiting per CPU information
(see other mail) I don't think removing this message by default
is a good idea. I regularly needed it for debugging some NUMA related
problems and they still happen moderately often even today.

I think the right approach here, to limit output, would be to figure out
a more compact output format, perhaps using a matrix in a table
or simply printing multiple pair per line.

-Andi

>
--
ak@... -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

Andi Kleen wrote:

> Mike Travis <travis@...> writes:
>
>> Limit number of SRAT messages of the form:
>> [    0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0
>
> While I generally agree on the concept of limiting per CPU information
> (see other mail) I don't think removing this message by default
> is a good idea. I regularly needed it for debugging some NUMA related
> problems and they still happen moderately often even today.
>
> I think the right approach here, to limit output, would be to figure out
> a more compact output format, perhaps using a matrix in a table
> or simply printing multiple pair per line.
>
> -Andi
>

On our UV systems, this really is redundant information and adds noise
to the console printout.  If you need to examine it, dmesg will provide
it (or don't use the limit_console_output flag)?

I had thought of some reduction techniques to reduce console output, but
it didn't seem worth the complexity.  Perhaps I was wrong?

Thanks,
Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

Andi Kleen wrote:

> Mike Travis <travis@...> writes:
>
>> Limit number of SRAT messages of the form:
>> [    0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0
>
> While I generally agree on the concept of limiting per CPU information
> (see other mail) I don't think removing this message by default
> is a good idea. I regularly needed it for debugging some NUMA related
> problems and they still happen moderately often even today.
>
> I think the right approach here, to limit output, would be to figure out
> a more compact output format, perhaps using a matrix in a table
> or simply printing multiple pair per line.
>
> -Andi
>

In this approach, I only print the first SRAT of each node...  Is
this closer to what you're looking for?

Thanks,
Mike

SGI x86_64 UV: Limit the number of number of SRAT messages

Limit number of SRAT messages of the form:

[    0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0

Cc: H. Peter Anvin <hpa@...>
Cc: x86@...
Cc: David Rientjes <rientjes@...>
Cc: Yinghai Lu <yinghai@...>
Cc: Mel Gorman <mel@...>
Cc: linux-kernel@...
Signed-off-by: Mike Travis <travis@...>
---
 arch/x86/mm/srat_64.c |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

--- linux.orig/arch/x86/mm/srat_64.c
+++ linux/arch/x86/mm/srat_64.c
@@ -115,6 +115,7 @@
 {
  int pxm, node;
  int apic_id;
+ static int last_node = -1;
 
  if (srat_disabled())
  return;
@@ -136,8 +137,11 @@
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+ if (node > last_node) {
+ printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
+       pxm, apic_id, node);
+ last_node = node;
+ }
 }
 
 /* Callback for Proximity Domain -> LAPIC mapping */
@@ -146,6 +150,7 @@
 {
  int pxm, node;
  int apic_id;
+ static int last_node = -1;
 
  if (srat_disabled())
  return;
@@ -170,8 +175,11 @@
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+ if (node > last_node) {
+ printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
+       pxm, apic_id, node);
+ last_node = node;
+ }
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[David Rientjes]

On Tue, 27 Oct 2009, Mike Travis wrote:

> --- linux.orig/arch/x86/mm/srat_64.c
> +++ linux/arch/x86/mm/srat_64.c
> @@ -115,6 +115,7 @@
> {
> int pxm, node;
> int apic_id;
> + static int last_node = -1;
>
> if (srat_disabled())
> return;
> @@ -136,8 +137,11 @@
> apicid_to_node[apic_id] = node;
> node_set(node, cpu_nodes_parsed);
> acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
> + if (node > last_node) {
> + printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> +       pxm, apic_id, node);
> + last_node = node;
> + }
> }
>
> /* Callback for Proximity Domain -> LAPIC mapping */
> @@ -146,6 +150,7 @@
> {
> int pxm, node;
> int apic_id;
> + static int last_node = -1;
>
> if (srat_disabled())
> return;
> @@ -170,8 +175,11 @@
> apicid_to_node[apic_id] = node;
> node_set(node, cpu_nodes_parsed);
> acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
> + if (node > last_node) {
> + printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> +       pxm, apic_id, node);
> + last_node = node;
> + }
> }
>
> #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
>

So on my Opteron I'll be getting this:

        SRAT: PXM 0 -> APIC 0 -> Node 0
        SRAT: PXM 1 -> APIC 2 -> Node 1

instead of this:

        SRAT: PXM 0 -> APIC 0 -> Node 0
        SRAT: PXM 0 -> APIC 1 -> Node 0
        SRAT: PXM 1 -> APIC 2 -> Node 1
        SRAT: PXM 1 -> APIC 3 -> Node 1

Do I need to infer what apic 1 or 3 map to with your patch (or whether
they are even valid)?

It would seem much better to print this information once the SRAT parsing
in acpi_numa_init() is complete and apicid_to_node[] is populated.  This
leads to the ideal beheavior, which is:

        SRAT: PXM 0 -> APIC {0-1} -> Node 0
        SRAT: PXM 1 -> APIC {2-3} -> Node 1

Something like the following patch?  (Regardless, we need to cc
linux-acpi@....  I've added it.)



x86: reduce srat verbosity in the kernel log

It's possible to reduce the number of SRAT messages emitted to the kernel
log by printing each valid pxm once and then creating bitmaps to represent
the apicids that map to the same node.

This reduces lines such as

        SRAT: PXM 0 -> APIC 0 -> Node 0
        SRAT: PXM 0 -> APIC 1 -> Node 0
        SRAT: PXM 1 -> APIC 2 -> Node 1
        SRAT: PXM 1 -> APIC 3 -> Node 1

to

        SRAT: PXM 0 -> APIC {0-1} -> Node 0
        SRAT: PXM 1 -> APIC {2-3} -> Node 1

Signed-off-by: David Rientjes <rientjes@...>
---
 arch/x86/mm/srat_64.c |   31 +++++++++++++++++++++++++++----
 drivers/acpi/numa.c   |    5 +++++
 include/linux/acpi.h  |    3 ++-
 3 files changed, 34 insertions(+), 5 deletions(-)

diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
--- a/arch/x86/mm/srat_64.c
+++ b/arch/x86/mm/srat_64.c
@@ -136,8 +136,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
 }
 
 /* Callback for Proximity Domain -> LAPIC mapping */
@@ -170,8 +168,33 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+}
+
+void __init acpi_numa_print_srat_mapping(void)
+{
+ DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC);
+ char apicid_list[MAX_LOCAL_APIC];
+ int i, j;
+
+ for (i = 0; i < MAX_PXM_DOMAINS; i++) {
+ int nid;
+
+ nid = pxm_to_node(i);
+ if (nid == NUMA_NO_NODE)
+ continue;
+
+ bitmap_zero(apicid_map, MAX_LOCAL_APIC);
+ for (j = 0; j < MAX_LOCAL_APIC; j++)
+ if (apicid_to_node[j] == nid)
+ set_bit(j, apicid_map);
+
+ if (bitmap_empty(apicid_map, MAX_LOCAL_APIC))
+ continue;
+ bitmap_scnlistprintf(apicid_list, MAX_LOCAL_APIC,
+     apicid_map, MAX_LOCAL_APIC);
+ pr_info("SRAT: PXM %u -> APIC {%s} -> Node %u\n",
+ i, apicid_list, nid);
+ }
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -281,6 +281,10 @@ acpi_table_parse_srat(enum acpi_srat_type id,
     handler, max_entries);
 }
 
+void __init __attribute__((weak)) acpi_numa_print_srat_mapping(void)
+{
+}
+
 int __init acpi_numa_init(void)
 {
  /* SRAT: Static Resource Affinity Table */
@@ -292,6 +296,7 @@ int __init acpi_numa_init(void)
  acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
       acpi_parse_memory_affinity,
       NR_NODE_MEMBLKS);
+ acpi_numa_print_srat_mapping();
  }
 
  /* SLIT: System Locality Information Table */
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -92,12 +92,13 @@ int acpi_table_parse_madt (enum acpi_madt_type id, acpi_table_entry_handler hand
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following six functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 void acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 void acpi_numa_arch_fixup(void);
+void acpi_numa_print_srat_mapping(void);
 
 #ifdef CONFIG_ACPI_HOTPLUG_CPU
 /* Arch dependent functions for cpu hotplug support */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

Hi David,

Very Cool, I'll try it out and let you know how it goes.

Note that it would be better to declare the BITMAP in the
static initdata section so it doesn't grow the stack by 4k
bytes.  (And it's thrown away after the kernel starts.)

Thanks,
Mike


David Rientjes wrote:

> On Tue, 27 Oct 2009, Mike Travis wrote:
>
>> --- linux.orig/arch/x86/mm/srat_64.c
>> +++ linux/arch/x86/mm/srat_64.c
>> @@ -115,6 +115,7 @@
>> {
>> int pxm, node;
>> int apic_id;
>> + static int last_node = -1;
>>
>> if (srat_disabled())
>> return;
>> @@ -136,8 +137,11 @@
>> apicid_to_node[apic_id] = node;
>> node_set(node, cpu_nodes_parsed);
>> acpi_numa = 1;
>> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
>> -       pxm, apic_id, node);
>> + if (node > last_node) {
>> + printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
>> +       pxm, apic_id, node);
>> + last_node = node;
>> + }
>> }
>>
>> /* Callback for Proximity Domain -> LAPIC mapping */
>> @@ -146,6 +150,7 @@
>> {
>> int pxm, node;
>> int apic_id;
>> + static int last_node = -1;
>>
>> if (srat_disabled())
>> return;
>> @@ -170,8 +175,11 @@
>> apicid_to_node[apic_id] = node;
>> node_set(node, cpu_nodes_parsed);
>> acpi_numa = 1;
>> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
>> -       pxm, apic_id, node);
>> + if (node > last_node) {
>> + printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
>> +       pxm, apic_id, node);
>> + last_node = node;
>> + }
>> }
>>
>> #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
>>
>
> So on my Opteron I'll be getting this:
>
> SRAT: PXM 0 -> APIC 0 -> Node 0
> SRAT: PXM 1 -> APIC 2 -> Node 1
>
> instead of this:
>
> SRAT: PXM 0 -> APIC 0 -> Node 0
> SRAT: PXM 0 -> APIC 1 -> Node 0
> SRAT: PXM 1 -> APIC 2 -> Node 1
> SRAT: PXM 1 -> APIC 3 -> Node 1
>
> Do I need to infer what apic 1 or 3 map to with your patch (or whether
> they are even valid)?
>
> It would seem much better to print this information once the SRAT parsing
> in acpi_numa_init() is complete and apicid_to_node[] is populated.  This
> leads to the ideal beheavior, which is:
>
> SRAT: PXM 0 -> APIC {0-1} -> Node 0
> SRAT: PXM 1 -> APIC {2-3} -> Node 1
>
> Something like the following patch?  (Regardless, we need to cc
> linux-acpi@....  I've added it.)
>
>
>
> x86: reduce srat verbosity in the kernel log
>
> It's possible to reduce the number of SRAT messages emitted to the kernel
> log by printing each valid pxm once and then creating bitmaps to represent
> the apicids that map to the same node.
>
> This reduces lines such as
>
> SRAT: PXM 0 -> APIC 0 -> Node 0
> SRAT: PXM 0 -> APIC 1 -> Node 0
> SRAT: PXM 1 -> APIC 2 -> Node 1
> SRAT: PXM 1 -> APIC 3 -> Node 1
>
> to
>
> SRAT: PXM 0 -> APIC {0-1} -> Node 0
> SRAT: PXM 1 -> APIC {2-3} -> Node 1
>
> Signed-off-by: David Rientjes <rientjes@...>
> ---
>  arch/x86/mm/srat_64.c |   31 +++++++++++++++++++++++++++----
>  drivers/acpi/numa.c   |    5 +++++
>  include/linux/acpi.h  |    3 ++-
>  3 files changed, 34 insertions(+), 5 deletions(-)
>
> diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
> --- a/arch/x86/mm/srat_64.c
> +++ b/arch/x86/mm/srat_64.c
> @@ -136,8 +136,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
>   apicid_to_node[apic_id] = node;
>   node_set(node, cpu_nodes_parsed);
>   acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
>  }
>  
>  /* Callback for Proximity Domain -> LAPIC mapping */
> @@ -170,8 +168,33 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
>   apicid_to_node[apic_id] = node;
>   node_set(node, cpu_nodes_parsed);
>   acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
> +}
> +
> +void __init acpi_numa_print_srat_mapping(void)
> +{
> + DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC);
> + char apicid_list[MAX_LOCAL_APIC];
> + int i, j;
> +
> + for (i = 0; i < MAX_PXM_DOMAINS; i++) {
> + int nid;
> +
> + nid = pxm_to_node(i);
> + if (nid == NUMA_NO_NODE)
> + continue;
> +
> + bitmap_zero(apicid_map, MAX_LOCAL_APIC);
> + for (j = 0; j < MAX_LOCAL_APIC; j++)
> + if (apicid_to_node[j] == nid)
> + set_bit(j, apicid_map);
> +
> + if (bitmap_empty(apicid_map, MAX_LOCAL_APIC))
> + continue;
> + bitmap_scnlistprintf(apicid_list, MAX_LOCAL_APIC,
> +     apicid_map, MAX_LOCAL_APIC);
> + pr_info("SRAT: PXM %u -> APIC {%s} -> Node %u\n",
> + i, apicid_list, nid);
> + }
>  }
>  
>  #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -281,6 +281,10 @@ acpi_table_parse_srat(enum acpi_srat_type id,
>      handler, max_entries);
>  }
>  
> +void __init __attribute__((weak)) acpi_numa_print_srat_mapping(void)
> +{
> +}
> +
>  int __init acpi_numa_init(void)
>  {
>   /* SRAT: Static Resource Affinity Table */
> @@ -292,6 +296,7 @@ int __init acpi_numa_init(void)
>   acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
>        acpi_parse_memory_affinity,
>        NR_NODE_MEMBLKS);
> + acpi_numa_print_srat_mapping();
>   }
>  
>   /* SLIT: System Locality Information Table */
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -92,12 +92,13 @@ int acpi_table_parse_madt (enum acpi_madt_type id, acpi_table_entry_handler hand
>  int acpi_parse_mcfg (struct acpi_table_header *header);
>  void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
>  
> -/* the following four functions are architecture-dependent */
> +/* the following six functions are architecture-dependent */
>  void acpi_numa_slit_init (struct acpi_table_slit *slit);
>  void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>  void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>  void acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
>  void acpi_numa_arch_fixup(void);
> +void acpi_numa_print_srat_mapping(void);
>  
>  #ifdef CONFIG_ACPI_HOTPLUG_CPU
>  /* Arch dependent functions for cpu hotplug support */

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Cyrill Gorcunov]

[David Rientjes - Tue, Oct 27, 2009 at 12:45:00PM -0700]
...
| On Tue, 27 Oct 2009, Mike Travis wrote:
| +
| +void __init acpi_numa_print_srat_mapping(void)
| +{
| + DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC);
| + char apicid_list[MAX_LOCAL_APIC];

Hi David, I suppose 32K on stack is too much :)
(perhaps gcc will move it out of stack?)

...

        -- Cyrill
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

Cyrill Gorcunov wrote:

> [David Rientjes - Tue, Oct 27, 2009 at 12:45:00PM -0700]
> ...
> | On Tue, 27 Oct 2009, Mike Travis wrote:
> | +
> | +void __init acpi_numa_print_srat_mapping(void)
> | +{
> | + DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC);
> | + char apicid_list[MAX_LOCAL_APIC];
>
> Hi David, I suppose 32K on stack is too much :)
> (perhaps gcc will move it out of stack?)
>
> ...
>
> -- Cyrill

Yeah, I missed that too on my first review.  (4k seems piddling
compared to 32k on the stack! ;-)

I moved them both to static and will be testing it shortly:

static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
static char apicid_list[MAX_LOCAL_APIC] __initdata;

Thanks,
Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[David Rientjes]

On Tue, 27 Oct 2009, Mike Travis wrote:

> Hi David,
>
> Very Cool, I'll try it out and let you know how it goes.
>
> Note that it would be better to declare the BITMAP in the
> static initdata section so it doesn't grow the stack by 4k
> bytes.  (And it's thrown away after the kernel starts.)
>

Right, here's an updated version.  I was thinking of MAX_PXM_DOMAINS being
256 instead of MAX_LOCAL_APIC :)

Here's an updated version.  apicid_map and apicid_list don't need to be
synchronized because there're no concurrency issues here on init.



x86: reduce srat verbosity in the kernel log

It's possible to reduce the number of SRAT messages emitted to the kernel
log by printing each valid pxm once and then creating bitmaps to represent
the apicids that map to the same node.

This reduces lines such as

        SRAT: PXM 0 -> APIC 0 -> Node 0
        SRAT: PXM 0 -> APIC 1 -> Node 0
        SRAT: PXM 1 -> APIC 2 -> Node 1
        SRAT: PXM 1 -> APIC 3 -> Node 1

to

        SRAT: PXM 0 -> APIC {0-1} -> Node 0
        SRAT: PXM 1 -> APIC {2-3} -> Node 1

Signed-off-by: David Rientjes <rientjes@...>
---
 arch/x86/mm/srat_64.c |   32 ++++++++++++++++++++++++++++----
 drivers/acpi/numa.c   |    5 +++++
 include/linux/acpi.h  |    3 ++-
 3 files changed, 35 insertions(+), 5 deletions(-)

diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
--- a/arch/x86/mm/srat_64.c
+++ b/arch/x86/mm/srat_64.c
@@ -36,6 +36,9 @@ static int num_node_memblks __initdata;
 static struct bootnode node_memblk_range[NR_NODE_MEMBLKS] __initdata;
 static int memblk_nodeid[NR_NODE_MEMBLKS] __initdata;
 
+static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
+static char apicid_list[MAX_LOCAL_APIC] __initdata;
+
 static __init int setup_node(int pxm)
 {
  return acpi_map_pxm_to_node(pxm);
@@ -136,8 +139,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
 }
 
 /* Callback for Proximity Domain -> LAPIC mapping */
@@ -170,8 +171,31 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
  apicid_to_node[apic_id] = node;
  node_set(node, cpu_nodes_parsed);
  acpi_numa = 1;
- printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
-       pxm, apic_id, node);
+}
+
+void __init acpi_numa_print_srat_mapping(void)
+{
+ int i, j;
+
+ for (i = 0; i < MAX_PXM_DOMAINS; i++) {
+ int nid;
+
+ nid = pxm_to_node(i);
+ if (nid == NUMA_NO_NODE)
+ continue;
+
+ bitmap_zero(apicid_map, MAX_LOCAL_APIC);
+ for (j = 0; j < MAX_LOCAL_APIC; j++)
+ if (apicid_to_node[j] == nid)
+ set_bit(j, apicid_map);
+
+ if (bitmap_empty(apicid_map, MAX_LOCAL_APIC))
+ continue;
+ bitmap_scnlistprintf(apicid_list, MAX_LOCAL_APIC,
+     apicid_map, MAX_LOCAL_APIC);
+ pr_info("SRAT: PXM %u -> APIC {%s} -> Node %u\n",
+ i, apicid_list, nid);
+ }
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -281,6 +281,10 @@ acpi_table_parse_srat(enum acpi_srat_type id,
     handler, max_entries);
 }
 
+void __init __attribute__((weak)) acpi_numa_print_srat_mapping(void)
+{
+}
+
 int __init acpi_numa_init(void)
 {
  /* SRAT: Static Resource Affinity Table */
@@ -292,6 +296,7 @@ int __init acpi_numa_init(void)
  acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
       acpi_parse_memory_affinity,
       NR_NODE_MEMBLKS);
+ acpi_numa_print_srat_mapping();
  }
 
  /* SLIT: System Locality Information Table */
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -92,12 +92,13 @@ int acpi_table_parse_madt (enum acpi_madt_type id, acpi_table_entry_handler hand
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following six functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 void acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 void acpi_numa_arch_fixup(void);
+void acpi_numa_print_srat_mapping(void);
 
 #ifdef CONFIG_ACPI_HOTPLUG_CPU
 /* Arch dependent functions for cpu hotplug support */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Cyrill Gorcunov]

[Mike Travis - Tue, Oct 27, 2009 at 01:23:42PM -0700]
>
> I moved them both to static and will be testing it shortly:
>
> static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
> static char apicid_list[MAX_LOCAL_APIC] __initdata;
>
> Thanks,
> Mike
>

Great! David has just updated patch too.

        -- Cyrill
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

I applied your previous patch with the change to use static and
here's the console output from a live system:


[    0.000000] SRAT: PXM 0 -> APIC {0-7,16-23} -> Node 0
[    0.000000] SRAT: PXM 1 -> APIC {32-39,48-55} -> Node 1
[    0.000000] SRAT: PXM 2 -> APIC {64-71,80-87} -> Node 2
[    0.000000] SRAT: PXM 3 -> APIC {96-103,112-119} -> Node 3
[    0.000000] SRAT: PXM 4 -> APIC {128-135,144-151} -> Node 4
[    0.000000] SRAT: PXM 5 -> APIC {160-167,176-183} -> Node 5
[    0.000000] SRAT: PXM 6 -> APIC {192-199,208-215} -> Node 6
[    0.000000] SRAT: PXM 7 -> APIC {224-231,240-247} -> Node 7
[    0.000000] SRAT: PXM 8 -> APIC {256-263,272-279} -> Node 8
[    0.000000] SRAT: PXM 9 -> APIC {288-295,304-311} -> Node 9
[    0.000000] SRAT: PXM 10 -> APIC {320-327,336-343} -> Node 10
[    0.000000] SRAT: PXM 11 -> APIC {352-359,368-375} -> Node 11
[    0.000000] SRAT: PXM 12 -> APIC {384-391,400-407} -> Node 12
[    0.000000] SRAT: PXM 13 -> APIC {416-423,432-439} -> Node 13
[    0.000000] SRAT: PXM 14 -> APIC {448-455,464-471} -> Node 14
[    0.000000] SRAT: PXM 15 -> APIC {480-487,496-503} -> Node 15
[    0.000000] SRAT: PXM 16 -> APIC {512-519,528-535} -> Node 16
[    0.000000] SRAT: PXM 17 -> APIC {544-551,560-567} -> Node 17
[    0.000000] SRAT: PXM 18 -> APIC {576-583,592-599} -> Node 18
[    0.000000] SRAT: PXM 19 -> APIC {608-615,624-631} -> Node 19
[    0.000000] SRAT: PXM 20 -> APIC {640-647,656-663} -> Node 20
[    0.000000] SRAT: PXM 21 -> APIC {672-679,688-695} -> Node 21
[    0.000000] SRAT: PXM 22 -> APIC {704-711,720-727} -> Node 22
[    0.000000] SRAT: PXM 23 -> APIC {736-743,752-759} -> Node 23  

                                                                                             
David Rientjes wrote:

> On Tue, 27 Oct 2009, Mike Travis wrote:
>
>> Hi David,
>>
>> Very Cool, I'll try it out and let you know how it goes.
>>
>> Note that it would be better to declare the BITMAP in the
>> static initdata section so it doesn't grow the stack by 4k
>> bytes.  (And it's thrown away after the kernel starts.)
>>
>
> Right, here's an updated version.  I was thinking of MAX_PXM_DOMAINS being
> 256 instead of MAX_LOCAL_APIC :)
>
> Here's an updated version.  apicid_map and apicid_list don't need to be
> synchronized because there're no concurrency issues here on init.
>
>
>
> x86: reduce srat verbosity in the kernel log
>
> It's possible to reduce the number of SRAT messages emitted to the kernel
> log by printing each valid pxm once and then creating bitmaps to represent
> the apicids that map to the same node.
>
> This reduces lines such as
>
> SRAT: PXM 0 -> APIC 0 -> Node 0
> SRAT: PXM 0 -> APIC 1 -> Node 0
> SRAT: PXM 1 -> APIC 2 -> Node 1
> SRAT: PXM 1 -> APIC 3 -> Node 1
>
> to
>
> SRAT: PXM 0 -> APIC {0-1} -> Node 0
> SRAT: PXM 1 -> APIC {2-3} -> Node 1
>
> Signed-off-by: David Rientjes <rientjes@...>
> ---
>  arch/x86/mm/srat_64.c |   32 ++++++++++++++++++++++++++++----
>  drivers/acpi/numa.c   |    5 +++++
>  include/linux/acpi.h  |    3 ++-
>  3 files changed, 35 insertions(+), 5 deletions(-)
>
> diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
> --- a/arch/x86/mm/srat_64.c
> +++ b/arch/x86/mm/srat_64.c
> @@ -36,6 +36,9 @@ static int num_node_memblks __initdata;
>  static struct bootnode node_memblk_range[NR_NODE_MEMBLKS] __initdata;
>  static int memblk_nodeid[NR_NODE_MEMBLKS] __initdata;
>  
> +static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
> +static char apicid_list[MAX_LOCAL_APIC] __initdata;
> +
>  static __init int setup_node(int pxm)
>  {
>   return acpi_map_pxm_to_node(pxm);
> @@ -136,8 +139,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
>   apicid_to_node[apic_id] = node;
>   node_set(node, cpu_nodes_parsed);
>   acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
>  }
>  
>  /* Callback for Proximity Domain -> LAPIC mapping */
> @@ -170,8 +171,31 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
>   apicid_to_node[apic_id] = node;
>   node_set(node, cpu_nodes_parsed);
>   acpi_numa = 1;
> - printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
> -       pxm, apic_id, node);
> +}
> +
> +void __init acpi_numa_print_srat_mapping(void)
> +{
> + int i, j;
> +
> + for (i = 0; i < MAX_PXM_DOMAINS; i++) {
> + int nid;
> +
> + nid = pxm_to_node(i);
> + if (nid == NUMA_NO_NODE)
> + continue;
> +
> + bitmap_zero(apicid_map, MAX_LOCAL_APIC);
> + for (j = 0; j < MAX_LOCAL_APIC; j++)
> + if (apicid_to_node[j] == nid)
> + set_bit(j, apicid_map);
> +
> + if (bitmap_empty(apicid_map, MAX_LOCAL_APIC))
> + continue;
> + bitmap_scnlistprintf(apicid_list, MAX_LOCAL_APIC,
> +     apicid_map, MAX_LOCAL_APIC);
> + pr_info("SRAT: PXM %u -> APIC {%s} -> Node %u\n",
> + i, apicid_list, nid);
> + }
>  }
>  
>  #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -281,6 +281,10 @@ acpi_table_parse_srat(enum acpi_srat_type id,
>      handler, max_entries);
>  }
>  
> +void __init __attribute__((weak)) acpi_numa_print_srat_mapping(void)
> +{
> +}
> +
>  int __init acpi_numa_init(void)
>  {
>   /* SRAT: Static Resource Affinity Table */
> @@ -292,6 +296,7 @@ int __init acpi_numa_init(void)
>   acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
>        acpi_parse_memory_affinity,
>        NR_NODE_MEMBLKS);
> + acpi_numa_print_srat_mapping();
>   }
>  
>   /* SLIT: System Locality Information Table */
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -92,12 +92,13 @@ int acpi_table_parse_madt (enum acpi_madt_type id, acpi_table_entry_handler hand
>  int acpi_parse_mcfg (struct acpi_table_header *header);
>  void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
>  
> -/* the following four functions are architecture-dependent */
> +/* the following six functions are architecture-dependent */
>  void acpi_numa_slit_init (struct acpi_table_slit *slit);
>  void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>  void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>  void acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
>  void acpi_numa_arch_fixup(void);
> +void acpi_numa_print_srat_mapping(void);
>  
>  #ifdef CONFIG_ACPI_HOTPLUG_CPU
>  /* Arch dependent functions for cpu hotplug support */

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[David Rientjes]

On Tue, 27 Oct 2009, Mike Travis wrote:

> I applied your previous patch with the change to use static and
> here's the console output from a live system:
>
>
> [    0.000000] SRAT: PXM 0 -> APIC {0-7,16-23} -> Node 0
> [    0.000000] SRAT: PXM 1 -> APIC {32-39,48-55} -> Node 1
> [    0.000000] SRAT: PXM 2 -> APIC {64-71,80-87} -> Node 2
> [    0.000000] SRAT: PXM 3 -> APIC {96-103,112-119} -> Node 3
> [    0.000000] SRAT: PXM 4 -> APIC {128-135,144-151} -> Node 4
> [    0.000000] SRAT: PXM 5 -> APIC {160-167,176-183} -> Node 5
> [    0.000000] SRAT: PXM 6 -> APIC {192-199,208-215} -> Node 6
> [    0.000000] SRAT: PXM 7 -> APIC {224-231,240-247} -> Node 7
> [    0.000000] SRAT: PXM 8 -> APIC {256-263,272-279} -> Node 8
> [    0.000000] SRAT: PXM 9 -> APIC {288-295,304-311} -> Node 9
> [    0.000000] SRAT: PXM 10 -> APIC {320-327,336-343} -> Node 10
> [    0.000000] SRAT: PXM 11 -> APIC {352-359,368-375} -> Node 11
> [    0.000000] SRAT: PXM 12 -> APIC {384-391,400-407} -> Node 12
> [    0.000000] SRAT: PXM 13 -> APIC {416-423,432-439} -> Node 13
> [    0.000000] SRAT: PXM 14 -> APIC {448-455,464-471} -> Node 14
> [    0.000000] SRAT: PXM 15 -> APIC {480-487,496-503} -> Node 15
> [    0.000000] SRAT: PXM 16 -> APIC {512-519,528-535} -> Node 16
> [    0.000000] SRAT: PXM 17 -> APIC {544-551,560-567} -> Node 17
> [    0.000000] SRAT: PXM 18 -> APIC {576-583,592-599} -> Node 18
> [    0.000000] SRAT: PXM 19 -> APIC {608-615,624-631} -> Node 19
> [    0.000000] SRAT: PXM 20 -> APIC {640-647,656-663} -> Node 20
> [    0.000000] SRAT: PXM 21 -> APIC {672-679,688-695} -> Node 21
> [    0.000000] SRAT: PXM 22 -> APIC {704-711,720-727} -> Node 22
> [    0.000000] SRAT: PXM 23 -> APIC {736-743,752-759} -> Node 23  
>

Quite the system you have there :)  What was once 760 lines has been
reduced to 24 without removing any information.

This seems to be the most we can reduce this particular output since we
don't support mapping multiple pxms to a single node.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Cyrill Gorcunov]

[David Rientjes - Tue, Oct 27, 2009 at 01:25:51PM -0700]
| On Tue, 27 Oct 2009, Mike Travis wrote:
|
...
| +
| +void __init acpi_numa_print_srat_mapping(void)
| +{
| + int i, j;
| +
| + for (i = 0; i < MAX_PXM_DOMAINS; i++) {
| + int nid;
| +
| + nid = pxm_to_node(i);
| + if (nid == NUMA_NO_NODE)

Btw, David, while you at it, I just curious -- shouldn't we test it
with NID_INVAL (as pxm_to_node_map initially defined to)? Not a big
deal at all (since they are both = -1) but for the record.
Or perhaps I miss something?

| + continue;
...

        -- Cyrill
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[David Rientjes]

On Tue, 27 Oct 2009, Cyrill Gorcunov wrote:

> | +void __init acpi_numa_print_srat_mapping(void)
> | +{
> | + int i, j;
> | +
> | + for (i = 0; i < MAX_PXM_DOMAINS; i++) {
> | + int nid;
> | +
> | + nid = pxm_to_node(i);
> | + if (nid == NUMA_NO_NODE)
>
> Btw, David, while you at it, I just curious -- shouldn't we test it
> with NID_INVAL (as pxm_to_node_map initially defined to)? Not a big
> deal at all (since they are both = -1) but for the record.
> Or perhaps I miss something?
>

I don't think we need to address that since NID_INVAL is going away and
will be replaced by NUMA_NO_NODE since Lee has exposed it globally in his
mempolicy patchset, and as you mention they are the same anyway.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Cyrill Gorcunov]

[David Rientjes - Tue, Oct 27, 2009 at 02:06:21PM -0700]
| On Tue, 27 Oct 2009, Cyrill Gorcunov wrote:
|
| > | +void __init acpi_numa_print_srat_mapping(void)
| > | +{
| > | + int i, j;
| > | +
| > | + for (i = 0; i < MAX_PXM_DOMAINS; i++) {
| > | + int nid;
| > | +
| > | + nid = pxm_to_node(i);
| > | + if (nid == NUMA_NO_NODE)
| >
| > Btw, David, while you at it, I just curious -- shouldn't we test it
| > with NID_INVAL (as pxm_to_node_map initially defined to)? Not a big
| > deal at all (since they are both = -1) but for the record.
| > Or perhaps I miss something?
| >
|
| I don't think we need to address that since NID_INVAL is going away and
| will be replaced by NUMA_NO_NODE since Lee has exposed it globally in his
| mempolicy patchset, and as you mention they are the same anyway.
|

I see. Thanks!

        -- Cyrill
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Mike Travis]

David Rientjes wrote:

> On Tue, 27 Oct 2009, Mike Travis wrote:
>
>> I applied your previous patch with the change to use static and
>> here's the console output from a live system:
>>
>>
>> [    0.000000] SRAT: PXM 0 -> APIC {0-7,16-23} -> Node 0
>> [    0.000000] SRAT: PXM 1 -> APIC {32-39,48-55} -> Node 1
>> [    0.000000] SRAT: PXM 2 -> APIC {64-71,80-87} -> Node 2
>> [    0.000000] SRAT: PXM 3 -> APIC {96-103,112-119} -> Node 3
>> [    0.000000] SRAT: PXM 4 -> APIC {128-135,144-151} -> Node 4
>> [    0.000000] SRAT: PXM 5 -> APIC {160-167,176-183} -> Node 5
>> [    0.000000] SRAT: PXM 6 -> APIC {192-199,208-215} -> Node 6
>> [    0.000000] SRAT: PXM 7 -> APIC {224-231,240-247} -> Node 7
>> [    0.000000] SRAT: PXM 8 -> APIC {256-263,272-279} -> Node 8
>> [    0.000000] SRAT: PXM 9 -> APIC {288-295,304-311} -> Node 9
>> [    0.000000] SRAT: PXM 10 -> APIC {320-327,336-343} -> Node 10
>> [    0.000000] SRAT: PXM 11 -> APIC {352-359,368-375} -> Node 11
>> [    0.000000] SRAT: PXM 12 -> APIC {384-391,400-407} -> Node 12
>> [    0.000000] SRAT: PXM 13 -> APIC {416-423,432-439} -> Node 13
>> [    0.000000] SRAT: PXM 14 -> APIC {448-455,464-471} -> Node 14
>> [    0.000000] SRAT: PXM 15 -> APIC {480-487,496-503} -> Node 15
>> [    0.000000] SRAT: PXM 16 -> APIC {512-519,528-535} -> Node 16
>> [    0.000000] SRAT: PXM 17 -> APIC {544-551,560-567} -> Node 17
>> [    0.000000] SRAT: PXM 18 -> APIC {576-583,592-599} -> Node 18
>> [    0.000000] SRAT: PXM 19 -> APIC {608-615,624-631} -> Node 19
>> [    0.000000] SRAT: PXM 20 -> APIC {640-647,656-663} -> Node 20
>> [    0.000000] SRAT: PXM 21 -> APIC {672-679,688-695} -> Node 21
>> [    0.000000] SRAT: PXM 22 -> APIC {704-711,720-727} -> Node 22
>> [    0.000000] SRAT: PXM 23 -> APIC {736-743,752-759} -> Node 23  
>>
>
> Quite the system you have there :)  What was once 760 lines has been
> reduced to 24 without removing any information.
>
> This seems to be the most we can reduce this particular output since we
> don't support mapping multiple pxms to a single node.

Yes, thanks very much for the optimization.

(And you can add my Acked-by or whatever you need.)

Tomorrow I will have more time on the system and will try out all the
new changes together, mostly with summarizing the Processor stats.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Andi Kleen-3]

>> Quite the system you have there :)  What was once 760 lines has been
>> reduced to 24 without removing any information.
>>
>> This seems to be the most we can reduce this particular output since we
>> don't support mapping multiple pxms to a single node.
>
> Yes, thanks very much for the optimization.
>
> (And you can add my Acked-by or whatever you need.)

Looks also good to me, thanks. Also Acked-by.

-Andi

--
ak@... -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Andi Kleen-3]

> +static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
> +static char apicid_list[MAX_LOCAL_APIC] __initdata;

Is MAX_LOCAL_APIC really big enough to print them all in ASCII?

It would be better to not use that large a buffer, but print
in smaller pieces (I realize this would enlarge your patch,
but then it would also save a lot of BSS)

-Andi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Yinghai Lu-5]

David Rientjes wrote:

> On Tue, 27 Oct 2009, Mike Travis wrote:
>
>> I applied your previous patch with the change to use static and
>> here's the console output from a live system:
>>
>>
>> [    0.000000] SRAT: PXM 0 -> APIC {0-7,16-23} -> Node 0
>> [    0.000000] SRAT: PXM 1 -> APIC {32-39,48-55} -> Node 1
>> [    0.000000] SRAT: PXM 2 -> APIC {64-71,80-87} -> Node 2
>> [    0.000000] SRAT: PXM 3 -> APIC {96-103,112-119} -> Node 3
>> [    0.000000] SRAT: PXM 4 -> APIC {128-135,144-151} -> Node 4
>> [    0.000000] SRAT: PXM 5 -> APIC {160-167,176-183} -> Node 5
>> [    0.000000] SRAT: PXM 6 -> APIC {192-199,208-215} -> Node 6
>> [    0.000000] SRAT: PXM 7 -> APIC {224-231,240-247} -> Node 7
>> [    0.000000] SRAT: PXM 8 -> APIC {256-263,272-279} -> Node 8
>> [    0.000000] SRAT: PXM 9 -> APIC {288-295,304-311} -> Node 9
>> [    0.000000] SRAT: PXM 10 -> APIC {320-327,336-343} -> Node 10
>> [    0.000000] SRAT: PXM 11 -> APIC {352-359,368-375} -> Node 11
>> [    0.000000] SRAT: PXM 12 -> APIC {384-391,400-407} -> Node 12
>> [    0.000000] SRAT: PXM 13 -> APIC {416-423,432-439} -> Node 13
>> [    0.000000] SRAT: PXM 14 -> APIC {448-455,464-471} -> Node 14
>> [    0.000000] SRAT: PXM 15 -> APIC {480-487,496-503} -> Node 15
>> [    0.000000] SRAT: PXM 16 -> APIC {512-519,528-535} -> Node 16
>> [    0.000000] SRAT: PXM 17 -> APIC {544-551,560-567} -> Node 17
>> [    0.000000] SRAT: PXM 18 -> APIC {576-583,592-599} -> Node 18
>> [    0.000000] SRAT: PXM 19 -> APIC {608-615,624-631} -> Node 19
>> [    0.000000] SRAT: PXM 20 -> APIC {640-647,656-663} -> Node 20
>> [    0.000000] SRAT: PXM 21 -> APIC {672-679,688-695} -> Node 21
>> [    0.000000] SRAT: PXM 22 -> APIC {704-711,720-727} -> Node 22
>> [    0.000000] SRAT: PXM 23 -> APIC {736-743,752-759} -> Node 23  
>>
>
> Quite the system you have there :)  What was once 760 lines has been
> reduced to 24 without removing any information.
>

can you change the apic to hex print?

YH
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[David Rientjes]

On Wed, 28 Oct 2009, Andi Kleen wrote:

> > +static DECLARE_BITMAP(apicid_map, MAX_LOCAL_APIC) __initdata;
> > +static char apicid_list[MAX_LOCAL_APIC] __initdata;
>
> Is MAX_LOCAL_APIC really big enough to print them all in ASCII?
>
> It would be better to not use that large a buffer, but print
> in smaller pieces (I realize this would enlarge your patch,
> but then it would also save a lot of BSS)
>

MAX_LOCAL_APIC was definitely an arbitrary choice here and has very little
relevance.  scnlistprintf will protect against overflow, but we still need
to decide upon a constant that will emit the most information possible
while not overly polluting the printk and saving on bss, as you mentioned.  
I suspect we could agree on a value as little as 128 and it would work for
the overwhelming majority (all?) of users.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/