Sun Feb 28 07:57:08 2021
EVENTS
 FREE
SOFTWARE
INSTITUTE

POLITICS
JOBS
MEMBERS'
CORNER

MAILING
LIST

NYLXS Mailing Lists and Archives
NYLXS Members have a lot to say and share but we don't keep many secrets. Join the Hangout Mailing List and say your peice.

DATE 2015-04-01

LEARN

2021-02-28 | 2021-01-28 | 2020-12-28 | 2020-11-28 | 2020-10-28 | 2020-09-28 | 2020-08-28 | 2020-07-28 | 2020-06-28 | 2020-05-28 | 2020-04-28 | 2020-03-28 | 2020-02-28 | 2020-01-28 | 2019-12-28 | 2019-11-28 | 2019-10-28 | 2019-09-28 | 2019-08-28 | 2019-07-28 | 2019-06-28 | 2019-05-28 | 2019-04-28 | 2019-03-28 | 2019-02-28 | 2019-01-28 | 2018-12-28 | 2018-11-28 | 2018-10-28 | 2018-09-28 | 2018-08-28 | 2018-07-28 | 2018-06-28 | 2018-05-28 | 2018-04-28 | 2018-03-28 | 2018-02-28 | 2018-01-28 | 2017-12-28 | 2017-11-28 | 2017-10-28 | 2017-09-28 | 2017-08-28 | 2017-07-28 | 2017-06-28 | 2017-05-28 | 2017-04-28 | 2017-03-28 | 2017-02-28 | 2017-01-28 | 2016-12-28 | 2016-11-28 | 2016-10-28 | 2016-09-28 | 2016-08-28 | 2016-07-28 | 2016-06-28 | 2016-05-28 | 2016-04-28 | 2016-03-28 | 2016-02-28 | 2016-01-28 | 2015-12-28 | 2015-11-28 | 2015-10-28 | 2015-09-28 | 2015-08-28 | 2015-07-28 | 2015-06-28 | 2015-05-28 | 2015-04-28 | 2015-03-28 | 2015-02-28 | 2015-01-28 | 2014-12-28 | 2014-11-28 | 2014-10-28

Key: Value:

Key: Value:

MESSAGE
DATE 2015-04-23
FROM Ruben Safir
SUBJECT Subject: [LIU Comp Sci] Fwd: Re: Kernel Scheduler and wiat queues
From owner-learn-outgoing-at-mrbrklyn.com Thu Apr 23 05:05:06 2015
Return-Path:
X-Original-To: archive-at-mrbrklyn.com
Delivered-To: archive-at-mrbrklyn.com
Received: by mrbrklyn.com (Postfix)
id 1ABC216116D; Thu, 23 Apr 2015 05:05:06 -0400 (EDT)
Delivered-To: learn-outgoing-at-mrbrklyn.com
Received: by mrbrklyn.com (Postfix, from userid 28)
id 08AB5161174; Thu, 23 Apr 2015 05:05:05 -0400 (EDT)
Delivered-To: learn-at-nylxs.com
Received: from mailbackend.panix.com (mailbackend.panix.com [166.84.1.89])
by mrbrklyn.com (Postfix) with ESMTP id C31B516116D
for ; Thu, 23 Apr 2015 05:04:40 -0400 (EDT)
Received: from [10.0.0.19] (www.mrbrklyn.com [96.57.23.82])
by mailbackend.panix.com (Postfix) with ESMTPSA id 6F6B212D3A;
Thu, 23 Apr 2015 05:04:39 -0400 (EDT)
Message-ID: <5538B5A7.2020102-at-panix.com>
Date: Thu, 23 Apr 2015 05:04:39 -0400
From: Ruben Safir
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: Ruben Safir ,
Maneesh Kongara ,
learn-at-nylxs.com
Subject: [LIU Comp Sci] Fwd: Re: Kernel Scheduler and wiat queues
References: <20150422091305.302-at-kylheku.com> <87sibsays5.fsf-at-doppelsaurus.mobileactivedefense.com> <20150422110344.469-at-kylheku.com> <20150422160337.511-at-kylheku.com>
In-Reply-To: <20150422160337.511-at-kylheku.com>
X-Forwarded-Message-Id: <20150422091305.302-at-kylheku.com> <87sibsays5.fsf-at-doppelsaurus.mobileactivedefense.com> <20150422110344.469-at-kylheku.com> <20150422160337.511-at-kylheku.com>
Content-Type: multipart/mixed;
boundary="------------050806050208030709040604"
Sender: owner-learn-at-mrbrklyn.com
Precedence: bulk
Reply-To: learn-at-mrbrklyn.com

This is a multi-part message in MIME format.
--------------050806050208030709040604
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit


--------------050806050208030709040604
Content-Type: message/rfc822;
name="Re: Kernel Scheduler and wiat queues.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: Kernel Scheduler and wiat queues.eml"

Path: reader1.panix.com!panix!goblin3!goblin2!goblin.stu.neva.ru!aioe.org!.POSTED!not-for-mail
From: Kaz Kylheku
Newsgroups: comp.unix.programmer
Subject: Re: Kernel Scheduler and wiat queues
Date: Wed, 22 Apr 2015 16:31:25 +0000 (UTC)
Organization: Aioe.org NNTP Server
Message-ID: <20150422091305.302-at-kylheku.com>
References:
NNTP-Posting-Host: OGJi3KNpFOhM58UHZwXj0w.user.speranza.aioe.org
X-Complaints-To: abuse-at-aioe.org
User-Agent: slrn/pre1.0.0-18 (Linux)
X-Notice: Filtered by postfilter v. 0.8.2
Xref: panix comp.unix.programmer:234522

On 2015-04-22, ruben safir wrote:
> DEFINE_WAIT(wait) //what IS wait EXACTLY in this context

A preprocessor token, used as the argument to a macro call.

To understand what it means, you must look at the macro expansion.

What the macro does is generate code to define a variable whose name is "wait".
It is abstracted away because, possibly, the variable is not only defined, but
requires initialization. It requires initialization in ways that could change
as the kernel is maintained, so it needs to be abstract.

> add_wait_queue(q, &wait); // in the current kernel this invovled
> // flag checking and a linked list
>
> while(!condition){ /* an event we are weighting for
> prepare_to_wait(&q, &wait, TASK_INTERRUPTIBLE);
> if(signal_pending(current))
> /* SIGNAl HANDLE */
> schedule();
> }
>
> finish_wait(&q, &wait);
>
> He also write how this proceeds to function and one part confuses me
>
> 5. When the taks awakens, it again checks whether the condition is
> true. If it is, it exists the loop. Otherwise it again calls schedule.

This is just the consequence of the code you have before you. When
the task awakens it returns out of the schedule() call, and continues
with the loop.

> This is not the order that it seems to follow according to the code.
>
> To me it looks like it should
> 1 - creat2 the wait queue

You really mean "wait queue node". The wait queue already exists, but
the task needs a queuing data structure so that it can add itself to
that queue.

(Why does a task need an extra structure for this rather than just some
link pointers inside its task structure. The answer is that a task
can wait on multiple wait queues queues at the same time:

DEFINE_WAIT(node0);
DEFINE_WAIT(node1);
/* ... */
DEFINE_WAIT(nodeN);

add_wait_queue(q0, &node0);
add_wait_queue(q1, &node1);
/* ... */
add_wait_queue(q1, &nodeN);

while (!condition) {
prepare_to_wait(&q0, &node0, TASK_INTERRUPTIBLE);
prepare_to_wait(&q1, &node1, TASK_INTERRUPTIBLE);
/* ... */
prepare_to_wait(&qn, &nodeN, TASK_INTERRUPTIBLE);

if(signal_pending(current)) { /* SIGNAl HANDLE */ }

schedule();
}

finish_wait(&q0, &node0);
finish_wait(&q1, &node1);
/* ... */
finish_wait(&qn, &nodeN);

> 5 check for signals ... notice the process is still moving. Does it
> stop and wait now?

No, it stops and waits somewhere inside the schedule() function, and only
there. Elsewhere, the code is running.

> 6 schedule itself on the runtime rbtree... which make NO sense unless
> there was a stopage I didn't know about.

It makes perfect sense. The task changes its status to "interruptible sleep",
and then informs the scheduler, by calling the schedule() function.
The scheduler puts that into effect, and removes the task from the run queue.
Then it makes a scheduling decision to dispatch a task. The task which
called it is not eligible for execution now, so another task is chosen.

> Isn't this reckless to leave this to users to write the code. Your
> begging for a race condition.

Yes, of course you are absolutely right; most kernel developers should not be
writing the above open code most of the time. The best justification for
it is that you're developing a new synchronization primitive (which itself
requires justification as to why the existing ones aren't good enough).

There are higher level primitives which wrap the above logic: semaphores,
read-write semaphores, mutexes, events, poll, ...

For instance, look for wait_event, wait_event_interruptible, and related
macros.

--------------050806050208030709040604
Content-Type: message/rfc822;
name="Re: Kernel Scheduler and wiat queues.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: Kernel Scheduler and wiat queues.eml"

Path: reader1.panix.com!panix!goblin3!goblin.stu.neva.ru!bolzen.all.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: Rainer Weikusat
Newsgroups: comp.unix.programmer
Subject: Re: Kernel Scheduler and wiat queues
Date: Wed, 22 Apr 2015 17:55:06 +0100
Message-ID: <87sibsays5.fsf-at-doppelsaurus.mobileactivedefense.com>
References: <20150422091305.302-at-kylheku.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Trace: individual.net HNSRaNth1zP/WQ5XpOOARwUPhN6pW+FgmsMxAm7H/2tNPOD+k=
Cancel-Lock: sha1:1KB3kgqkkr0Bkv0gGl9sQmE8FnE= sha1:+g4tgBpuw3aST93HN5A71CajRIA=
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux)
Xref: panix comp.unix.programmer:234523

Kaz Kylheku writes:
> On 2015-04-22, ruben safir wrote:

[...]

>> 6 schedule itself on the runtime rbtree... which make NO sense unless
>> there was a stopage I didn't know about.
>
> It makes perfect sense. The task changes its status to "interruptible sleep",
> and then informs the scheduler, by calling the schedule() function.
> The scheduler puts that into effect, and removes the task from the run
> queue.

schedule() is the top-level entry point of the scheduler, so, a more
accurate description would be "it calls into the scheduler in order to
make that select some task to run". Maybe a higher priority task than
the one which called schedule is runnable. In this case, the higher
priority task will run instead of the one which called
schedule. Otherwise, this task will continue to run unless it changed
its state to something other than 'runnable' first.

>> Isn't this reckless to leave this to users to write the code. Your
>> begging for a race condition.
>
> Yes, of course you are absolutely right; most kernel developers should not be
> writing the above open code most of the time. The best justification for
> it is that you're developing a new synchronization primitive (which itself
> requires justification as to why the existing ones aren't good
> enough).

I completely disagree with this: Just because more or less useful and
more or less buggy of bug-free 'special-case wait for this, that or
something else' code keeps piling up in in include/linux/wait.h doesn't
mean anyone ought to be required to read through all of this whenever
some kernel code has to wait for something to select the currently least
worse bit of infrastructure code from there, considering that the simple
solution of just coding the wait loop will work for every situation and
is all over the kernel for historical reasons, anyway.

Some people can't ever do anything without producing a "general purpose
library" (or a couple of these) [or can't ever do anything except
that]. Considering this obvious lack of practically applicable skills,
no code deserves to be used just because someone wrote it.


--------------050806050208030709040604
Content-Type: message/rfc822;
name="Re: Kernel Scheduler and wiat queues.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: Kernel Scheduler and wiat queues.eml"

Path: reader1.panix.com!panix!goblin2!goblin.stu.neva.ru!aioe.org!.POSTED!not-for-mail
From: Kaz Kylheku
Newsgroups: comp.unix.programmer
Subject: Re: Kernel Scheduler and wiat queues
Date: Wed, 22 Apr 2015 18:20:48 +0000 (UTC)
Organization: Aioe.org NNTP Server
Message-ID: <20150422110344.469-at-kylheku.com>
References:
<20150422091305.302-at-kylheku.com>
<87sibsays5.fsf-at-doppelsaurus.mobileactivedefense.com>
NNTP-Posting-Host: OGJi3KNpFOhM58UHZwXj0w.user.speranza.aioe.org
X-Complaints-To: abuse-at-aioe.org
User-Agent: slrn/pre1.0.0-18 (Linux)
X-Notice: Filtered by postfilter v. 0.8.2
Xref: panix comp.unix.programmer:234524

On 2015-04-22, Rainer Weikusat wrote:
> Kaz Kylheku writes:
>> On 2015-04-22, ruben safir wrote:
>>> Isn't this reckless to leave this to users to write the code. Your
>>> begging for a race condition.
>>
>> Yes, of course you are absolutely right; most kernel developers should not be
>> writing the above open code most of the time. The best justification for
>> it is that you're developing a new synchronization primitive (which itself
>> requires justification as to why the existing ones aren't good
>> enough).
>
> I completely disagree with this: Just because more or less useful and
> more or less buggy of bug-free 'special-case wait for this, that or
> something else' code keeps piling up in in include/linux/wait.h doesn't
> mean anyone ought to be required to read through all of this whenever
> some kernel code has to wait for something to select the currently least
> worse bit of infrastructure code from there, considering that the simple
> solution of just coding the wait loop will work for every situation and
> is all over the kernel for historical reasons, anyway.

I'm the opposite. Give me a large number of primitives to choose from!

By the way, I used mutexes and condition variables in the kernel back in 1998,
before any such a thing existed in the kernel, and made a little package of
that. It supported a driver I was working on:

http://www.kylheku.com/~kaz/lmc.html

A few years ago, on another job, I also wrote a "super poll function".
With this bugger, you could, simultaneously:

- wait on a set of sockets (struct socket *)
- wait on a set of files (struct file *)
- wait on a condition variable
- with a timeout

with this mega-function, I multiplexed a ton of functionality onto a single
kernel thread.

--------------050806050208030709040604
Content-Type: message/rfc822;
name="Re: Kernel Scheduler and wiat queues.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: Kernel Scheduler and wiat queues.eml"

Path: reader1.panix.com!panix!goblin3!goblin2!goblin.stu.neva.ru!feeder.erje.net!1.eu.feeder.erje.net!news2.arglkargh.de!news.mixmin.net!aioe.org!.POSTED!not-for-mail
From: Kaz Kylheku
Newsgroups: comp.unix.programmer
Subject: Re: Kernel Scheduler and wiat queues
Date: Wed, 22 Apr 2015 23:04:32 +0000 (UTC)
Organization: Aioe.org NNTP Server
Message-ID: <20150422160337.511-at-kylheku.com>
References:
<20150422091305.302-at-kylheku.com>
<87sibsays5.fsf-at-doppelsaurus.mobileactivedefense.com>
<20150422110344.469-at-kylheku.com>
NNTP-Posting-Host: OGJi3KNpFOhM58UHZwXj0w.user.speranza.aioe.org
X-Complaints-To: abuse-at-aioe.org
User-Agent: slrn/pre1.0.0-18 (Linux)
X-Notice: Filtered by postfilter v. 0.8.2
Xref: panix comp.unix.programmer:234525

On 2015-04-22, Kaz Kylheku wrote:
> By the way, I used mutexes and condition variables in the kernel back in 1998,
> before any such a thing existed in the kernel, and made a little package of
> that. It supported a driver I was working on:
>
> http://www.kylheku.com/~kaz/lmc.html
>
> A few years ago, on another job, I also wrote a "super poll function".
> With this bugger, you could, simultaneously:
>
> - wait on a set of sockets (struct socket *)
> - wait on a set of files (struct file *)
> - wait on a condition variable
> - with a timeout

I just noticed that I had added that function to the above "LMC" project, haha.

--------------050806050208030709040604--

  1. 2015-04-01 Ruben <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] external paths
  2. 2015-04-01 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] omp pthread madness
  3. 2015-04-02 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Fw: Perl Developer / Linux SysAdmin Opportunities in South Florida
  4. 2015-04-02 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: omp pthread madness
  5. 2015-04-02 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: omp pthread madness
  6. 2015-04-02 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: [New discussion] which technologies/skills are good to have
  7. 2015-04-02 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: omp pthread madness
  8. 2015-04-05 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: Depths and heights
  9. 2015-04-09 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: Kernel thread scheduling
  10. 2015-04-09 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: Kernel thread scheduling
  11. 2015-04-13 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] BASh Shell Scripting
  12. 2015-04-13 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Chapter 5 Monitor
  13. 2015-04-13 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: [Perlweekly] #194 - The CPAN PR Challenge Now Has a Mini-Me!
  14. 2015-04-13 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Kernel Hacking Resources
  15. 2015-04-14 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Invitation: Vintage Computer Festival
  16. 2015-04-14 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] hashing slots sizes
  17. 2015-04-15 Ruben <mrbrklyn-at-panix.com> Re: [LIU Comp Sci] BASh Shell Scripting
  18. 2015-04-15 Maneesh Kongara <maneeshkongara-at-gmail.com> Re: [LIU Comp Sci] BASh Shell Scripting
  19. 2015-04-15 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Monday: Join 121 HackerNestologists at "HackerNest NYC April
  20. 2015-04-15 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: You're confirmed for: NYLUG Open hacker hours
  21. 2015-04-15 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Hash Tables Examples
  22. 2015-04-15 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: hashing quadradic probes
  23. 2015-04-15 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] [ruben-at-mrbrklyn.com: Re: Heap Management Problem]
  24. 2015-04-16 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Re: virtualbox
  25. 2015-04-16 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] linsched
  26. 2015-04-16 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] scheduler project thus far
  27. 2015-04-17 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Come Sunday Morning
  28. 2015-04-17 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Kernel Scheduling Project
  29. 2015-04-17 Ruben <ruben.safir-at-my.liu.edu> Subject: [LIU Comp Sci] more advanced coding musings ...
  30. 2015-04-19 Tony Genao <tony.genao-at-my.liu.edu> Re: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  31. 2015-04-19 Ruben Safir <mrbrklyn-at-panix.com> Re: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  32. 2015-04-19 Tony Genao <tony.genao-at-my.liu.edu> Re: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  33. 2015-04-19 Ruben <mrbrklyn-at-panix.com> Re: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  34. 2015-04-19 Tony Genao <tony.genao-at-my.liu.edu> Re: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  35. 2015-04-19 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: [NYLXS - HANGOUT] Summers Here
  36. 2015-04-19 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: The more I learn the less I know
  37. 2015-04-22 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Re: wait queues semiphores kernel implementations
  38. 2015-04-22 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] linux locks and mutex
  39. 2015-04-23 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Re: Kernel Scheduler and wiat queues
  40. 2015-04-25 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Kernel Scheduling and wait queues
  41. 2015-04-27 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: OS Cpt 9 27 question
  42. 2015-04-27 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Fwd: Tomorrow: You and 30 other Flatirons are going to "Flatiron
  43. 2015-04-28 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: DMCA exemption commenting process broken beyond repair
  44. 2015-04-29 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Do you have notes for this question
  45. 2015-04-29 Ruben Safir <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Professional Troll job available
  46. 2015-04-29 Ruben <mrbrklyn-at-panix.com> Subject: [LIU Comp Sci] Re: DMCA exemption commenting process broken beyond repair

NYLXS are Do'ers and the first step of Doing is Joining! Join NYLXS and make a difference in your community today!